v0.5.3

[linhr]

Overview

You can find the release summary in the documentation.

The PySail Python package is available on PyPI.

What's Changed

• deps(npm): bump the all group with 2 updates by @dependabot[bot] in deps(npm): bump the all group with 2 updates #1453
• deps(cargo): bump the all group with 5 updates by @dependabot[bot] in deps(cargo): bump the all group with 5 updates #1454
• fix: expand wildcard to column references for COUNT(DISTINCT *) by @davidlghellin in fix: expand wildcard to column references for COUNT(DISTINCT *)  #1394
• chore: upgrade rand, rand_chacha, and rand_distr by @Copilot in chore: upgrade rand, rand_chacha, and rand_distr #1457
• chore: upgrade kube and k8s-openapi by @Copilot in chore: upgrade kube and k8s-openapi #1458
• chore: bdd style testing guidence for Copilot agent by @lonless9 in chore: bdd style testing guidence for Copilot agent #1463
• chore: bump aws-lc-sys from 0.37.0 to 0.37.1 in the cargo group across 1 directory by @dependabot[bot] in chore: bump aws-lc-sys from 0.37.0 to 0.37.1 in the cargo group across 1 directory #1467
• feat: emit execution metrics on a configured interval by @Copilot in feat: emit execution metrics on a configured interval #1465
• feat: support serialization for CatalogCommand by @Copilot in feat: support serialization for CatalogCommand #1466
• feat: add the system table to expose AppConfig by @Copilot in feat: add the system table to expose AppConfig #1464
• feat: manage functions in catalog manager by @linhr in feat: manage functions in catalog manager #1471
• fix: distributed planner for SortPreservingMergeExec by @linhr in fix: distributed planner for SortPreservingMergeExec #1473
• chore: bump svgo from 4.0.0 to 4.0.1 in the npm_and_yarn group across 1 directory by @dependabot[bot] in chore: bump svgo from 4.0.0 to 4.0.1 in the npm_and_yarn group across 1 directory #1475
• fix: update schema reference in pushdown filter for Delta Lake by @lonless9 in fix: update schema reference in pushdown filter for Delta Lake #1477
• fix: using SQL with Python data sources by @shehabgamin in fix: using SQL with Python data sources #1469
• chore: bump MSRV to 1.91.0 by @Copilot in chore: bump MSRV to 1.91.0 #1476
• refactor: align the log scan subtree with table provider scan for Delta Lake by @lonless9 in refactor: align the log scan subtree with table provider scan for Delta Lake #1346
• fix: correct SQL syntax for CREATE DATABASE by @linhr in fix: correct SQL syntax for CREATE DATABASE #1480
• feat: move catalog commands to physical plans by @Copilot in feat: move catalog commands to physical plans #1479
• fix: namespace handling for catalog providers by @linhr in fix: namespace handling for catalog providers #1484
• fix: truncate gRPC error messages to prevent RESOURCE_EXHAUSTED by @davidlghellin in fix: truncate gRPC error messages to prevent RESOURCE_EXHAUSTED #1488
• fix: support non-literal expressions in window RANGE frame boundaries by @davidlghellin in fix: support non-literal expressions in window RANGE frame boundaries #1482
• fix: ensure CollectLeft HashJoinExec build side has single partition by @davidlghellin in fix: ensure CollectLeft HashJoinExec build side has single partition #1483
• feat: resolve default table location using database location by @Copilot in feat: resolve default table location using database location #1492
• deps(npm): bump the all group with 6 updates by @dependabot[bot] in deps(npm): bump the all group with 6 updates #1497
• deps(cargo): bump the all group with 10 updates by @dependabot[bot] in deps(cargo): bump the all group with 10 updates #1498
• refactor: move BDD steps to pysail.testing and remove DataFrame BDD tests by @Copilot in refactor: move BDD steps to pysail.testing and remove DataFrame BDD tests #1496
• refactor: restructure compatibility check utilities and tests by @Copilot in refactor: restructure compatibility check utilities and tests #1495
• refactor: internalization for Delta Lake by @lonless9 in refactor: internalization for Delta Lake #1461
• chore: store syrupy snapshots in snapshots directories instead of __snapshots__ by @Copilot in chore: store syrupy snapshots in snapshots directories instead of __snapshots__ #1501
• chore: bump quinn-proto from 0.11.13 to 0.11.14 in the cargo group across 1 directory by @dependabot[bot] in chore: bump quinn-proto from 0.11.13 to 0.11.14 in the cargo group across 1 directory #1503
• refactor: reorganize snapshot testing files by @Copilot in refactor: reorganize snapshot testing files #1502
• fix: update snapshot file locations by @Copilot in fix: update snapshot file locations #1505
• refactor: move test utilities to pysail.testing.spark.utils by @Copilot in refactor: move test utilities to pysail.testing.spark.utils #1506
• fix: move delta metadata pruning to a post-replay stats pipeline by @Copilot in fix: move delta metadata pruning to a post-replay stats pipeline #1504
• fix: use stable Delta log file schema for JSON-only log reads by @Copilot in fix: use stable Delta log file schema for JSON-only log reads #1500
• chore: cache mtime by @lonless9 in chore: cache mtime #1513
• feat: set up bdd testing & implement partition transforms for Iceberg by @lonless9 in feat: set up bdd testing & implement partition transforms for Iceberg #1183
• chore: update dependencies by @lonless9 in chore: update dependencies #1515
• chore: revert cache mtime by @lonless9 in chore: revert cache mtime #1516
• chore: bump lz4_flex from 0.12.0 to 0.12.1 in the cargo group across 1 directory by @dependabot[bot] in chore: bump lz4_flex from 0.12.0 to 0.12.1 in the cargo group across 1 directory #1529
• deps(cargo): bump the all group with 4 updates by @dependabot[bot] in deps(cargo): bump the all group with 4 updates #1527
• feat: version checksum file write support for Delta Lake by @lonless9 in feat: version checksum file write support for Delta Lake #1530
• fix: saveAsTable with mode="append" creates table when it does not exist by @Copilot in fix: saveAsTable with mode="append" creates table when it does not exist #1525
• feat: version checksum file read support for Delta Lake by @lonless9 in feat: version checksum file read support for Delta Lake #1532
• fix: support df.write.format(...).option("path", path).save() by @Copilot in fix: support df.write.format(...).option("path", path).save() #1533
• fix: soundex Utf8View support, LargeUtf8 handling, and edge cases by @davidlghellin in fix: soundex Utf8View support, LargeUtf8 handling, and edge cases #1517
• feat: add Spark 4.1 functions for TIME type by @tamirkifle in feat: add Spark 4.1 functions for TIME type #1470
• feat: add CLI command to run PySpark script by @linhr in feat: add CLI command to run PySpark script #1535
• test: add TPC-H and TPC-DS parity tests by @linhr in test: add TPC-H and TPC-DS parity tests #1509
• fix: use structured partition fields for Iceberg by @lonless9 in fix: use structured partition fields for Iceberg #1512
• fix: array_min/max NULL input and float-to-Decimal128 overflow by @davidlghellin in fix: array_min/max NULL input and float-to-Decimal128 overflow #1536
• feat: streamline log segment resolution for Delta Lake by @lonless9 in feat: streamline log segment resolution for Delta Lake #1534
• docs: agent skills and function support by @linhr in docs: agent skills and function support #1544
• chore: prepare v0.5.3 by @linhr in chore: prepare v0.5.3 #1542
• chore: prepare v0.5.3 by @linhr in chore: prepare v0.5.3 #1546
• chore: prepare v0.5.3 by @linhr in chore: prepare v0.5.3 #1547

Full Changelog: v0.5.2...v0.5.3

---

This discussion was created from the release v0.5.3.