Skip to content

CHANGELOG.md

Latest commit

 

History

History
343 lines (166 loc) · 25.4 KB

CHANGELOG.md

File metadata and controls

343 lines (166 loc) · 25.4 KB

Changelog

1.1.0 (2026-04-06)

Features

  • Add support for %dpip magic to install packages on Spark session (#176) (e335569)
  • Auto-detect more popular IDEs for usage tracking (#182) (51c6f92)

Documentation

1.0.2 (2026-02-04)

Bug Fixes

  • Improve the error message when Application Default Credentials are not configured (#173) (4a5be44)

1.0.1 (2025-12-05)

Bug Fixes

  • Switch to pyspark package as a dependency (#170) (9b0e306)

1.0.0 (2025-12-04)

Bug Fixes

  • Add validation when location or project ID are not provided (#168) (8da16ed)

1.0.0rc7 (2025-11-13)

Features

  • Depend on the lightweight Spark Connect client (#166) (89d77af)
  • support sharable session termination by adding terminate option to stop() function (#162) (3beb0fc)

Bug Fixes

  • address the error display issue on workbench only (#154) (538b7ca)
  • Do not manually set already pre-defined Spark BQ Connector configs (#165) (3403ce7)
  • Do not print stacktrace for known errors in all IPython envs (#159) (c693b63)
  • error (stack trace) propagation in non-Colab IPython environments (#158) (1988c42)
  • Remove 'Dataproc Session' link from the cell output and update 'View Session Details' button url. (#163) (43b696c)
  • Remove redundant empty/non-actionable per-operation Spark UI link and progress bar (#157) (7059ecc)
  • Update Python version mapping for Dataproc runtime 3.0 to Python 3.12 (#155) (16e98e3)
  • Use CLI tqdm if ipywidgets not installed (#167) (a0d2551)

1.0.0rc6 (2025-10-10)

Features

  • Add support for running Spark Connect client inside the Dataproc s8s batch (#150) (e5b1708)
  • add support for sparksql magic (#137) (274f013)

Bug Fixes

  • Ensure DataprocSparkConnectException displays error messages in all Jupyter environments (#136) (1fc26f5)
  • pysparkvalue error (#144) (436d97e)
  • Revert the exception message fix in SC 1.0 (#147) (21b6239)

1.0.0rc5 (2025-09-19)

Bug Fixes

  • Automatically set authentication type to SERVICE_ACCOUNT when a service account is provided (#139) (ee1d237)

1.0.0rc4 (2025-08-28)

Features

  • Add custom session ID support and validation for DataprocSparkSession (#132) (112574d)

Bug Fixes

  • Fix session creation display logic and session details button (#134) (cd6f813)

1.0.0rc3 (2025-08-21)

Features

  • Add server/client runtime version compatibility check (#128) (065cb50)

Bug Fixes

  • Fix per-operation Spark UI link display logic (#131) (547b2fe)

1.0.0rc2 (2025-08-18)

Features

  • Per-operation execution progress bar (#129) (ba68104)

1.0.0rc1 (2025-08-05)

⚠ BREAKING CHANGES

  • Upgrade to Spark 4 client (#111)

Features

0.9.0 (2025-07-31)

Features

  • add builder methods for session configuration (#115) (3a6a218)
  • Add Python version check and warning (#114) (65e8d4d)
  • Add view session details button (#123) (01b8f05)
  • Refactor and centralize client environment detection (#117) (3b1a0a0)

Bug Fixes

  • Detect only IDE (not plugins) in the environment.py (#125) (4733981)

0.8.3 (2025-07-21)

Bug Fixes

  • Update per-operation Spark UI URL print logic to skip SELECT clause (#118) (c01f9a5)

0.8.2 (2025-07-08)

Bug Fixes

  • Set SPARK_CONNECT_MODE_ENABLED env var (#112) (f3e0b7a)

0.8.1 (2025-07-01)

Features

  • Hyperlink Session URL on Session creation (#105) (58d3cdb)

Bug Fixes

  • Don't show IPython display object for non-interactive IPython Environment (#106) (832d018)
  • Fix unit tests providing needed mock dependencies (#110) (d6e2ed6)
  • Hide 'View Session Details' button from the output. (#107) (a36df90)

0.8.0 (2025-06-13)

Features

  • Add 'View Session Details' button on Session create. (#88) (54e42b7)
  • Add colab-notebook-id label to Dataproc Spark Connect session (#94) (66883f7)
  • Per-operation Spark UI URL (#91) (09392c7)

Bug Fixes

  • Extract Colab notebook ID from full path (#103) (c209798)
  • Validate Colab notebook ID for Dataproc session labels (#104) (d9fbf51)

0.7.5 (2025-05-27)

Bug Fixes

  • Pin PySpark dependency until new version is supported (#92) (0d238f3)

0.7.4 (2025-05-16)

Features

  • Add default datasource config via env var (#87) (71c3c1a)

0.7.3 (2025-05-08)

Bug Fixes

  • change default runtime version to 2.3 (#82) (7796cfe)
  • use event to stop pbar thread (#80) (14c9e01)

0.7.2 (2025-04-28)

Bug Fixes

  • stop session creation progress bar on cell termination (#78) (ebf71c8)

0.7.1 (2025-04-28)

Bug Fixes

  • remove version validation for SC client/server (#76) (7c9a4e6)

0.7.0 (2025-04-24)

Features

  • display progress bar during session creation (#70) (bd74d93)
  • Migrate to env vars from the config file for defaults configuration (#73) (5be4355)

Bug Fixes

  • fix unit test run from hanging indefinitely (#71) (eb78141)
  • improve log and output messages consistency (#74) (6558119)
  • minor code cleanup and output fixes (#68) (474e15c)

0.6.0 (2025-04-22)

⚠ BREAKING CHANGES

  • rename code and Python package to dataproc-spark-connect (#66)

Code Refactoring

  • rename code and Python package to dataproc-spark-connect (#66) (aa58789)

0.5.6 (2025-03-13)

Bug Fixes

  • return error message in render_traceback (#60) (9bc71d7)

0.5.5 (2025-03-12)

Bug Fixes

  • Check if the s8s session is terminated before creating connection (#55) (9f7e806)

0.5.4 (2025-03-10)

Bug Fixes

  • catch PermissionDenied errors while creating session (#53) (b0905cf)
  • integration test table cleanup (#58) (755de6d)
  • only use error message to throw RuntimeError (#51) (1f9136e)
  • suppress stack trace for InvalidArgument and PermissionDenied (#56) (87e5eb0)
  • unit test expected error message (#54) (e43c8bd)

0.5.3 (2025-02-28)

Bug Fixes

  • fixed a bug in the TCP-over-websocket proxy where one end of the connection might not be closed when the other end is closed. (932cb48)
  • Switch pyspark dependency to pyspark[connect] (#43) (23aec22)

Documentation

0.5.2 (2025-02-11)

Bug Fixes

  • append project to session URIs and use location instead of the region in the builder (#40) (b7134d9)

0.5.1 (2025-02-05)

Bug Fixes

  • Support Spark Connect Server URIs w/o trailing / (23dff94)

0.5.0 (2025-02-04)

⚠ BREAKING CHANGES

  • rename dataprocConfig to googleSessionConfig

Bug Fixes

  • rename dataprocConfig to googleSessionConfig (f041e9d)

0.4.1 (2025-01-31)

Bug Fixes

  • remove import reference of old package (#31) (28651cb)

0.4.0 (2025-01-29)

⚠ BREAKING CHANGES

  • rename python package to google-spark-connect (#25)
  • Rename package to google.cloud.spark_connect.GoogleSparkSession

Bug Fixes

  • Remove unused/invalid "spark" field in session config proto (d349b15)
  • terminate s8s session on kernel termination (#24) (beeaa98)

Code Refactoring

  • rename package and class to google.cloud.spark_connect.GoogleSparkSession (#21) (313dba4)
  • rename python package to google-spark-connect (#25) (357d1fe)

0.2.0 (2024-12-05)

Miscellaneous Chores

0.1.0 (2024-12-05)

Bug Fixes

  • Rename /regions/ path to /locations/ (8d7e4c8)
  • Rename /regions/ path to /locations/ in client (#13) (f51151a)