Pre-Release of Polars 2.0

Polars has released a pre-release candidate for version 2.0, with the full release expected in the coming weeks. The primary objective of the major version update is to eliminate past design decisions that currently limit functionality, as well as to change default settings to more sensible options that will benefit a wider user base.

The most significant change is that all LazyFrame queries will now default to using the streaming engine, which is anticipated to result in up to five times faster performance and reduced memory usage for casual Polars users. A migration guide has been provided to assist users in transitioning to this major update.

The increase in strictness is intended to catch errors early in the data processing pipeline, preventing potentially misleading results. For instance, when using the is_in expression on different data types, Polars will now raise an InvalidOperationError instead of silently casting data types, as was the case in previous versions. This improvement can help avoid issues related to lossy data type conversions, such as the example provided with user_id and flagged_ids.

In addition to these changes, Polars 2.0 also introduces stricter behavior for horizontal concatenation, now checking lengths instead of silently filling with null values. This ensures that the intention is more explicit, and users must opt-in to padding when desired. Furthermore, many previously ambiguous or unnecessary casts have been removed, with clearer alternatives provided to parse data more accurately.

Examples include the use of .cat.to(dtype) for int to categorical conversion and .str.to_date() / .str.to_datetime() for converting string data to date or datetime formats.

To help users adapt to these changes, two new exceptions have been added: polars.exceptions.AttributeRemovedError and polars.exceptions.ArgumentRemovedError. These exceptions provide more explicit error messages and guide users to the updated API. Most of the deprecated functionality has been phased out over time, and users who have kept their pipelines up-to-date should not encounter significant issues.

If users believe that certain functionality should be retained, they are encouraged to reach out to the Polars development team.