'h6jSrSSKJr SSKrSSKrSSKrSSKJr SSKJ r J r J r J r J r SSKJrJr SSKJr SSKJrJrJrJrJrJrJrJrJrJr SSKJr SS K!J"r# SS K$J%r%J&r&J'r' SS K(J)r)J*r*J+r+J,r,J-r-J.r.J/r/J0r0 SS K1J2r2 SS K3J4r4J5r5J6r6 SSK7J8r8 SSK9J:r: SSK;JJ?r? SSK@JArAJBrB SSKCJDrDJErEJFrFJGrGJHrHJIrIJJrJJKrK SSKLJMrMJNrNJOrO SSKPJQrQ SSKRJSrSJTrTJUrU SSKVJWrW SSKXJYrYJZrZJ[r[J\r\J]r]J^r^J_r_J`r`JaraJbrbJcrcJdrdJere SSKfJgrg SSKhJiriJjrjJkrkJlrlJmrmJnrnJoroJprpJqrqJrrrJsrsJtrt SSKhJurv SSKhJwrx SSKhJyrz SSK{J|r|J}r}J~r~JrJr SSKJrJr SS KJr SS!KJr SS"KJrJr \GR"\5 SS#KJr SS$KJr SS%KJr SSS5 \(GaSSKrSS&KJrJrJ r SS'KJr SS(KJr SS)KJr SSKrSSKrSSKJr SSKrSS*KrJr SS+KJr SS,KJr SS-K!JrJrJrJr SS.K$JrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJr SS/KCJr SS0KJr SS1KJr SS2KJr \GRS3:a SS4KJrJr OSS4KJrJr \GRS5:aSS6KJ5r5 OSS6KJ5r5 \"S75r\"S85r"S9S:5rSz4Module containing logic related to eager DataFrames.) annotationsN) defaultdict) GeneratorIterableMappingSequenceSized)BytesIOStringIO)Path) IO TYPE_CHECKINGAnyCallableClassVarNoReturnTypeVarcastget_argsoverload) functions) DbWriteMode JaxExportTypeTorchExportType) arrow_to_pydfdataframe_to_pydf dict_to_pydfiterable_to_pydf numpy_to_pydfpandas_to_pydfsequence_to_pydfseries_to_pydf)parse_as_duration_string)deprecate_renamed_parameter deprecatedissue_deprecation_warningget_df_item_by_key)parse_into_expression) is_pycapsulepycapsule_to_frame)serialize_polars_object)issue_unstable_warningunstable)is_bool_sequence no_defaultnormalize_filepath parse_versionqualified_type_namerequire_same_type scale_byteswarn_null_comparison) wrap_exprwrap_ldfwrap_s)NotebookFormatter)DynamicGroupByGroupByRollingGroupBy) DataFramePlot) N_INFER_DEFAULTBooleanFloat32Float64Int32Int64NullObjectStringStructUInt16UInt32UInt64)INTEGER_DTYPES) _ALTAIR_AVAILABLE_GREAT_TABLES_AVAILABLE_PANDAS_AVAILABLE_PYARROW_AVAILABLE_check_for_numpy_check_for_pandas_check_for_pyarrow_check_for_torchaltair great_tablesimport_optionaltorch)numpy)pandas)pyarrow)ColumnNotFoundErrorInvalidOperationErrorModuleUpgradeRequiredErrorNoRowsReturnedErrorTooManyRowsReturnedError)collit) CompatLevel)Schema)_expand_selector_dicts_expand_selectors) PyDataFrame)dtype_str_repr)write_clipboard_string) CollectionIteratorr) timedelta)IOBase)Literal)GT)Workbook) Worksheet)DataTypeExpr LazyFrameSeries)-AsofJoinStrategyAvroCompressionClosedIntervalColumnFormatDictColumnNameOrSelectorColumnTotalsDefinitionColumnWidthsDefinitionComparisonOperatorConditionalFormatDictConnectionOrCursor CsvQuoteStyle DbWriteEngine EngineTypeFillNullStrategyFrameInitTypes IndexOrderIntoExprIntoExprColumnIpcCompression JoinStrategyJoinValidationLabelMaintainOrderJoinMultiColSelectorMultiIndexSelectorOneOrMoreDataTypes OrientationParquetCompressionParquetMetadataPartitioningSchemePivotAggPolarsDataTypePythonDataTypeQuantileMethodRowTotalsDefinitionSchemaDefinition SchemaDict SelectorTypeSerializationFormatSingleColSelectorSingleIndexSelectorSizeUnitStartByUniqueKeepStrategyUnstackDirection) NoDefaultPolarsDataFrame)CredentialProviderFunction PolarsDataset) ) Concatenate ParamSpec)r )r%TPc\rSrSr%SrS\S'SS1rS\S'GSgS S S \S S .GShS jjjr\ SS.GSiSjj5r \ GSjSj5r \ GSkS S S.GSlSjjj5r \ GSkS S S S S.GSmSjjj5r GSnSjr\ GSoSj5r\\"5GSpSj55r\\"5GSqSj55r\GSrSj5r\GSsSj5r\GSsSj5r\GStSj5r\R0GSuSj5r\GSvS j5r\GSwS!j5r\GSxS"j5rGSgGSyS#jjrGSzGS{S$jjrGS|S%jrGS}S&jrGS|S'jr GS~S(jr!GSS)jr"GSS*jr#GSS+jr$GSS,jr%GSS-jr&GSS.jr'GSS/jr(GSS0jr)GSS1jr*GSS2jr+GSS3jr,GSS4jr-GSS5jr.GSS6jr/GSS7jr0GSS8jr1GSS9jr2GSS:jr3GSS;jr4GSS<jr5GSS=jr6GSS>jr7GSS?jr8\9GSS@j5r:\9GSSAj5r:\9GSSBj5r:GSSCjr:GSSDjr;GSsSEjrGStSHjr?GSkGSSIjjr@S SJ.GSSKjjrAGSxSLjrBGSgGSSMjjrC\D"SNSOSPSQ9S SR.GSSSjj5rE\9STSU.GSSVjj5rF\9GSSWj5rF\9GSSXj5rFS SU.GSSYjjrFGSSZjrGS[S S S S S\.GSS]jjrH\9GSSTSTSTSTSTS^.GSS_jjj5rI\9STSTSTSTSTS^.GSS`jj5rI\"5GSS S S S S[S^.GSSajjj5rI\9GSSTSTSTSb.GSScjjj5rJ\9STSTSTSb.GSSdjj5rJ\9STSTSTSb.GSSejj5rJ\"5GSS S S Sb.GSSfjjj5rJS Sg.GSShjjrKGSSijrLGSSjjrMGSGSSkjjrNGSGSSljjrO\9GSSTS.GSSmjjj5rP\9GSGSSnjj5rP\9STS.GSSojj5rPGSkSS.GSSpjjjrP\9GSGSSqjj5rQ\9GSSrj5rQGSkGSSsjjrQ\9GSkGSStjj5rR\9GSSuj5rRGSkGSSvjjrR\9GSkSTSTSTSTSTSTSTSTSTSTSTSTSTSTSTSTSTSw.GSSxjjj5rS\9STSTSTSTSTSTSTSTSTSTSTSTSTSTSTSTSTSw.GSSyjj5rSGSkS S SzS{S|S}S S S S S S S S S S~SSw.GSSjjjrSSS.GSSjjrTGSGSSjjrUGSgSS S S S S S S S S S S S SS S S S S S S S.GSSjjjrV\9SS S S~SS.GSSjj5rW\9SS S S~SS.GSSjj5rW\D"SNSOSPSQ9SS S S~SS.GSSjj5rW\9SS S.GSSjj5rX\9SS S.GSSjj5rX\D"SNSOSPSQ9SS S.GSSjj5rXSS S S S S S S SS S~SS S S.GSSjjrYSS S S.GSSjjrZ\"5GSSj5r[\9STSTSTSTSTS.GSSjj5r\\9STSTSTS.GSSjj5r\SS S S~S S S.GSSjjr\GSGSSjjr]S SS S.GSSjjr^GSSjr_S S.GSSjjr`GSSjraGSSjrbGSSjrc\9STSTSTS.GSSjj5rd\9STSTS.GSSjj5rd\9STSTS.GSSjj5rdSSS S.GSSjjrdGSSS.GSSjjjreGSSjrfGSSjrgS S S S S.GSSjjrhSS.GSSjjri\D"SSSSQ9S S.GSSjj5rj\D"SSSSQ9S S.GSSjj5rkS S.GSSjjrlGSkGSSjjrmGSGSSjjrnGSGSSjjroGSGSSjjrpGSkGSSjjrqGSkGSSjjrrGSSjrsGSSjrtGSGSSjjru\v"S5GSGSSjj5rwS S.GSSjjrx\D"SSSSQ9S SS S.GSSjj5ry\D"SSSSQ9S S S SSS SS.GSSjj5rz\D"SSSSQ9S S S.GSSjj5r{S S S S S S SSS S S S S S S.GSSjjr|\D"SSSSQ9GSS S SSS S S S.GSSjjj5r}\"5SS.GSSjj5r~GSkSS.GSSjjjrS S.GSSjjrS S.GSSjjrGSSjrS S.GSSjjrGSSjrS S.GSSjjrGSGSSjjrGSSjrGSSjr\9STS.GSSjj5r\9GSSj5r\S.GSSjjrGSS S.GSSjjjrGSSjrGSSjr\D"SSSSQ9S S S S S GSGS.GSGSjj5rGSkS S S GS.GSGSjjjrGSS S GS.GSGSjjr\9STSTSTGS.GSGS jj5r\9STSTGS .GSGS jj5r\9STSTGS .GSGS jj5rS S S GS.GSGS jjrGSS GS.GSGSjjjrGSGSjrGSGSjrGSGSjrGSGSjrGSGSjrGSGSjrGSGSjr\9GSGSGSjj5r\9GSGSj5rGSGSGSjjrGSGSjrGSGSjrGSGSjrGSGSjrGSGSjrS GS.GSGS jjrGSGS!jrS GS.GSGS"jjrGSGSGS#jjrGSGSGS$jjrGSGS%jrGSGS&jrGS GS GS'jjrGSkGSS S GS(.GS GS)jjjrGSkGS*S GS+.GS GS,jjjrGSkGS GS-jjr\v"GS.5GSGS/j5rGSGS0jrGSGS1jrGSkS S S S GS2.GSGS3jjjrGSGS4jr\9GSSTSTGS5.GSGS6jjj5r\9GSSTGS7.GSGS8jjj5rGSkS S GS5.GSGS9jjjr\9STGS:.GSGS;jj5r\9GSGS<j5rS GS:.GSGS=jjr\9STSTSTGS>.GSGS?jj5r\9STSTGS@.GSGSAjj5r\9STSTGSB.GSGSCjj5r\9STGSD.GSGSEjj5rS S S GS>.GSGSFjjr\9STSTGSG.GSGSHjj5r\9STGSI.GSGSJjj5rS GSKGSG.GSGSLjjrGSGSMjrGSGSGSNjjrS S.GS GSOjjrGSGS!GSPjjrGS"GS#GSQjjrGSGSRjrGS$GSSjrGS%GSGSTjjrGS&GSUjrGS'GSVjrGS(GSWjrS GSX.GS)GSYjjr\"5GS*S S S SGSZ.GS+GS[jjj5rGSGS\jr\v"GS]5GS,GS-GS^jj5r\"5GS_GS_GS_GS_GS`GS`GSa.GS.GSbjj5rGSgGS/GScjjrS S S GSd.GS0GSejjrGSfrg (1 DataFrameu Two-dimensional data structure representing data as a table with rows and columns. Parameters ---------- data : dict, Sequence, ndarray, Series, or pandas.DataFrame Two-dimensional data in various forms; dict input must contain Sequences, Generators, or a `range`. Sequence may contain Series or other Sequences. schema : Sequence of str, (str,DataType) pairs, or a {str:DataType,} dict The schema of the resulting DataFrame. The schema may be declared in several ways: * As a dict of {name:type} pairs; if type is None, it will be auto-inferred. * As a list of column names; in this case types are automatically inferred. * As a list of (name,type) pairs; this is equivalent to the dictionary form. If you supply a list of column names that does not match the names in the underlying data, the names given here will overwrite them. The number of names given in the schema should match the underlying data dimensions. If set to `None` (default), the schema is inferred from the data. schema_overrides : dict, default None Support type specification or override of one or more columns; note that any dtypes inferred from the schema param will be overridden. The number of entries in the schema should match the underlying data dimensions, unless a sequence of dictionaries is being passed, in which case a *partial* schema can be declared to prevent specific fields from being loaded. strict : bool, default True Throw an error if any `data` value does not exactly match the given or inferred data type for that column. If set to `False`, values that do not match the data type are cast to that data type or, if casting is not possible, set to null instead. orient : {'col', 'row'}, default None Whether to interpret two-dimensional data as columns or as rows. If None, the orientation is inferred by matching the columns and data dimensions. If this does not yield conclusive results, column orientation is used. infer_schema_length : int or None The maximum number of rows to scan for schema inference. If set to `None`, the full data may be scanned *(this can be slow)*. This parameter only applies if the input data is a sequence or generator of rows; other input is read as-is. nan_to_null : bool, default False If the data comes from one or more numpy arrays, can optionally convert input data np.nan values to null instead. This is a no-op for all other input data. Notes ----- Polars explicitly does not support subclassing of its core data types. See the following GitHub issue for possible workarounds: https://github.com/pola-rs/polars/issues/2846#issuecomment-1711799869 Examples -------- Constructing a DataFrame from a dictionary: >>> data = {"a": [1, 2], "b": [3, 4]} >>> df = pl.DataFrame(data) >>> df shape: (2, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 3 │ │ 2 ┆ 4 │ └─────┴─────┘ Notice that the dtypes are automatically inferred as polars Int64: >>> df.dtypes [Int64, Int64] To specify a more detailed/specific frame schema you can supply the `schema` parameter with a dictionary of (name,dtype) pairs... >>> data = {"col1": [0, 2], "col2": [3, 7]} >>> df2 = pl.DataFrame(data, schema={"col1": pl.Float32, "col2": pl.Int64}) >>> df2 shape: (2, 2) ┌──────┬──────┐ │ col1 ┆ col2 │ │ --- ┆ --- │ │ f32 ┆ i64 │ ╞══════╪══════╡ │ 0.0 ┆ 3 │ │ 2.0 ┆ 7 │ └──────┴──────┘ ...a sequence of (name,dtype) pairs... >>> data = {"col1": [1, 2], "col2": [3, 4]} >>> df3 = pl.DataFrame(data, schema=[("col1", pl.Float32), ("col2", pl.Int64)]) >>> df3 shape: (2, 2) ┌──────┬──────┐ │ col1 ┆ col2 │ │ --- ┆ --- │ │ f32 ┆ i64 │ ╞══════╪══════╡ │ 1.0 ┆ 3 │ │ 2.0 ┆ 4 │ └──────┴──────┘ ...or a list of typed Series. >>> data = [ ... pl.Series("col1", [1, 2], dtype=pl.Float32), ... pl.Series("col2", [3, 4], dtype=pl.Int64), ... ] >>> df4 = pl.DataFrame(data) >>> df4 shape: (2, 2) ┌──────┬──────┐ │ col1 ┆ col2 │ │ --- ┆ --- │ │ f32 ┆ i64 │ ╞══════╪══════╡ │ 1.0 ┆ 3 │ │ 2.0 ┆ 4 │ └──────┴──────┘ Constructing a DataFrame from a numpy ndarray, specifying column names: >>> import numpy as np >>> data = np.array([(1, 2), (3, 4)], dtype=np.int64) >>> df5 = pl.DataFrame(data, schema=["a", "b"], orient="col") >>> df5 shape: (2, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 3 │ │ 2 ┆ 4 │ └─────┴─────┘ Constructing a DataFrame from a list of lists, row orientation specified: >>> data = [[1, 2, 3], [4, 5, 6]] >>> df6 = pl.DataFrame(data, schema=["a", "b", "c"], orient="row") >>> df6 shape: (2, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ 1 ┆ 2 ┆ 3 │ │ 4 ┆ 5 ┆ 6 │ └─────┴─────┴─────┘ rg_dfplotstylezClassVar[set[str]] _accessorsNTF)schema_overridesstrictorientinfer_schema_length nan_to_nullc  Uc[0X#S9Ulg[U[5(a[UUUUUS9Ulg[U[[ [ 45(a[UUUUUUUS9Ulg[U[R5(a[XX4S9Ulg[U5(a3[U[R5(a[UUUUUUS9Ulg[U5(a/[U[ R"5(a[%XX4S9Ulg['U5(a/[U[(R*5(a[-XX4S9Ulg[/U5(a@[U[0R25(a![UR5SS9UUUUUS9Ulg[7US5(dD[U[85(d/[U[:[<45(a[?UUUUUUS 9Ulg[U[R*5(a[AXX4S9Ulg[CU5(aUUUS9RUlgS [GU5RH<S 3n[KU5e) N)schemar)rrrr)rrrrrr)rrr)rrrrrF)force__arrow_c_stream__)rrrrrz3DataFrame constructor called with unsupported type z for the `data` parameter)&rr isinstancedictlisttuplerr!plrur"rQnpndarrayrrSpaTablerrRpdrr rTrXTensorrYhasattrr rrrrr*r+type__name__ TypeError) selfdatarrrrrrmsgs iC:\Users\julio\OneDrive\Documentos\Trabajo\Ideas Frescas\venv\Lib\site-packages\polars/dataframe/frame.py__init__DataFrame.__init__gsK <#6DHd # ##!1' DHtUH5 6 6'!1$7'DHbii ( (%6FDHd # # 4(D(D$!1' DH  % %*T288*D*D$6FDHt $ $D",,)G)G%6FDHd # # 4(F(F$  '!1' DH233tU++4)X!677'!1$7 DHbll + +(6FDH$  )!1c HFd4jFYFYE\+, C. binary)formatct[U[5(a([UR5R 55nO&[U[ [ 45(a [U5nUS:Xa[RnO(US:Xa[RnOSU<3n[U5eURU"U55$)uJ Read a serialized DataFrame from a file. Parameters ---------- source Path to a file or a file-like object (by file-like object, we refer to objects that have a `read()` method, such as a file handler (e.g. via builtin `open` function) or `BytesIO`). format The format with which the DataFrame was serialized. Options: - `"binary"`: Deserialize from binary format (bytes). This is the default. - `"json"`: Deserialize from JSON format (string). See Also -------- DataFrame.serialize Notes ----- Serialization is not stable across Polars versions: a LazyFrame serialized in one Polars version may not be deserializable in another Polars version. Examples -------- >>> import io >>> df = pl.DataFrame({"a": [1, 2, 3], "b": [4.0, 5.0, 6.0]}) >>> bytes = df.serialize() >>> pl.DataFrame.deserialize(io.BytesIO(bytes)) shape: (3, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ f64 │ ╞═════╪═════╡ │ 1 ┆ 4.0 │ │ 2 ┆ 5.0 │ │ 3 ┆ 6.0 │ └─────┴─────┘ rjson0`format` must be one of {'binary', 'json'}, got ) rr r getvalueencodestrr r1rgdeserialize_binarydeserialize_json ValueError _from_pydf)clssourcer deserializerrs r deserializeDataFrame.deserializesZ fh ' 'V__.5578F d , ,'/F X &99L v &77LFvjQCS/ !~~l6233rc4URU5nXlU$)z7Construct Polars DataFrame from FFI PyDataFrame object.)__new__r)rpy_dfdfs rrDataFrame._from_pydf s[[  r)rrechunkc 8UR[UUUUS95$)a Construct a DataFrame from an Arrow table. This operation will be zero copy for the most part. Types that are not supported by Polars may be cast to the closest supported type. Parameters ---------- data : arrow Table, RecordBatch, or sequence of sequences Data representing an Arrow Table or RecordBatch. schema : Sequence of str, (str,DataType) pairs, or a {str:DataType,} dict The DataFrame schema may be declared in several ways: * As a dict of {name:type} pairs; if type is None, it will be auto-inferred. * As a list of column names; in this case types are automatically inferred. * As a list of (name,type) pairs; this is equivalent to the dictionary form. If you supply a list of column names that does not match the names in the underlying data, the names given here will overwrite them. The number of names given in the schema should match the underlying data dimensions. schema_overrides : dict, default None Support type specification or override of one or more columns; note that any dtypes inferred from the columns param will be overridden. rechunk : bool, default True Make sure that all data is in contiguous memory. )rrr)rr)rrrrrs r _from_arrowDataFrame._from_arrows*F~~ !1    r)rrr include_indexc <UR[UUUUUUS95$)a Construct a Polars DataFrame from a pandas DataFrame. Parameters ---------- data : pandas DataFrame Two-dimensional data represented as a pandas DataFrame. schema : Sequence of str, (str,DataType) pairs, or a {str:DataType,} dict The DataFrame schema may be declared in several ways: * As a dict of {name:type} pairs; if type is None, it will be auto-inferred. * As a list of column names; in this case types are automatically inferred. * As a list of (name,type) pairs; this is equivalent to the dictionary form. If you supply a list of column names that does not match the names in the underlying data, the names given here will overwrite them. The number of names given in the schema should match the underlying data dimensions. schema_overrides : dict, default None Support type specification or override of one or more columns; note that any dtypes inferred from the columns param will be overridden. rechunk : bool, default True Make sure that all data is in contiguous memory. nan_to_null : bool, default True If the data contains NaN values they will be converted to null/None. include_index : bool, default False Load any non-default pandas indexes as columns. )rrrrr)rr )rrrrrrrs r _from_pandasDataFrame._from_pandas=s0L~~ !1'+    rcolumncPURRXR5 U$)z,Replace a column by a new Series (in place).)rreplace_s)rr new_columns r_replaceDataFrame._replacens / rcLUR[R"X55$N)rrg_import_columns)rpointerwidths rrDataFrame._import_columnsss~~k99'IJJrc[(a[[R5S:a Sn[ U5e[ U5$)a Create a plot namespace. .. warning:: This functionality is currently considered **unstable**. It may be changed at any point without it being considered a breaking change. .. versionchanged:: 1.6.0 In prior versions of Polars, HvPlot was the plotting backend. If you would like to restore the previous plotting functionality, all you need to do is add `import hvplot.polars` at the top of your script and replace `df.plot` with `df.hvplot`. Polars does not implement plotting logic itself, but instead defers to `Altair `_: - `df.plot.line(**kwargs)` is shorthand for `alt.Chart(df).mark_line(tooltip=True).encode(**kwargs).interactive()` - `df.plot.point(**kwargs)` is shorthand for `alt.Chart(df).mark_point(tooltip=True).encode(**kwargs).interactive()` (and `plot.scatter` is provided as an alias) - `df.plot.bar(**kwargs)` is shorthand for `alt.Chart(df).mark_bar(tooltip=True).encode(**kwargs).interactive()` - for any other attribute `attr`, `df.plot.attr(**kwargs)` is shorthand for `alt.Chart(df).mark_attr(tooltip=True).encode(**kwargs).interactive()` For configuration, we suggest reading `Chart Configuration `_. For example, you can: - Change the width/height/title with ``.properties(width=500, height=350, title="My amazing plot")``. - Change the x-axis label rotation with ``.configure_axisX(labelAngle=30)``. - Change the opacity of the points in your scatter plot with ``.configure_point(opacity=.5)``. Examples -------- Scatter plot: >>> df = pl.DataFrame( ... { ... "length": [1, 4, 6], ... "width": [4, 5, 6], ... "species": ["setosa", "setosa", "versicolor"], ... } ... ) >>> df.plot.point(x="length", y="width", color="species") # doctest: +SKIP Set the x-axis title by using ``altair.X``: >>> import altair as alt >>> df.plot.point( ... x=alt.X("length", title="Length"), y="width", color="species" ... ) # doctest: +SKIP Line plot: >>> from datetime import date >>> df = pl.DataFrame( ... { ... "date": [date(2020, 1, 2), date(2020, 1, 3), date(2020, 1, 4)] * 2, ... "price": [1, 4, 6, 1, 5, 2], ... "stock": ["a", "a", "a", "b", "b", "b"], ... } ... ) >>> df.plot.line(x="date", y="price", color="stock") # doctest: +SKIP Bar plot: >>> df = pl.DataFrame( ... { ... "day": ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"] * 2, ... "group": ["a"] * 7 + ["b"] * 7, ... "value": [1, 3, 2, 4, 5, 6, 1, 1, 3, 2, 4, 5, 1, 2], ... } ... ) >>> df.plot.bar( ... x="day", y="value", color="day", column="group" ... ) # doctest: +SKIP Or, to make a stacked version of the plot above: >>> df.plot.bar(x="day", y="value", color="group") # doctest: +SKIP )rz%altair>=5.4.0 is required for `.plot`)rMr2rU __version__r^r>rrs rrDataFrame.plotws8x! M&2D2D$E $Q9C,S1 1T""rc^[(d Sn[U5e[R"U5$)a Create a Great Table for styling. .. warning:: This functionality is currently considered **unstable**. It may be changed at any point without it being considered a breaking change. Polars does not implement styling logic itself, but instead defers to the Great Tables package. Please see the `Great Tables reference `_ for more information and documentation. Examples -------- Import some styling helpers, and create example data: >>> import polars.selectors as cs >>> from great_tables import loc, style >>> df = pl.DataFrame( ... { ... "site_id": [0, 1, 2], ... "measure_a": [5, 4, 6], ... "measure_b": [7, 3, 3], ... } ... ) Emphasize the site_id as row names: >>> df.style.tab_stub(rowname_col="site_id") # doctest: +SKIP Fill the background for the highest measure_a value row: >>> df.style.tab_style( ... style.fill("yellow"), ... loc.body(rows=pl.col("measure_a") == pl.col("measure_a").max()), ... ) # doctest: +SKIP Put a spanner (high-level label) over measure columns: >>> df.style.tab_spanner( ... "Measures", cs.starts_with("measure") ... ) # doctest: +SKIP Format measure_b values to two decimal places: >>> df.style.fmt_number("measure_b", decimals=2) # doctest: +SKIP z%great_tables is required for `.style`)rNModuleNotFoundErrorrVrors rrDataFrame.styles*b'&9C%c* *t$$rc6URR5$)zx Get the shape of the DataFrame. Examples -------- >>> df = pl.DataFrame({"foo": [1, 2, 3, 4, 5]}) >>> df.shape (5, 1) )rshapers rr DataFrame.shapesxx~~rc6URR5$)z Get the number of rows. Returns ------- int Examples -------- >>> df = pl.DataFrame({"foo": [1, 2, 3, 4, 5]}) >>> df.height 5 )rheightr s rrDataFrame.heightsxx  rc6URR5$)z Get the number of columns. Returns ------- int Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [4, 5, 6], ... } ... ) >>> df.width 2 )rrr s rrDataFrame.width-s(xx~~rc6URR5$)u^ Get or set column names. Returns ------- list of str A list containing the name of each column in order. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.columns ['foo', 'bar', 'ham'] Set column names: >>> df.columns = ["apple", "banana", "orange"] >>> df shape: (3, 3) ┌───────┬────────┬────────┐ │ apple ┆ banana ┆ orange │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═══════╪════════╪════════╡ │ 1 ┆ 6 ┆ a │ │ 2 ┆ 7 ┆ b │ │ 3 ┆ 8 ┆ c │ └───────┴────────┴────────┘ )rcolumnsr s rrDataFrame.columnsCsJxx!!rc:URRU5 g)z Change the column names of the `DataFrame`. Parameters ---------- names A list with new names for the `DataFrame`. The length of the list should be equal to the width of the `DataFrame`. N)rset_column_names)rnamess rrrjs !!%(rc6URR5$)u" Get the column data types. The data types can also be found in column headers when printing the DataFrame. Returns ------- list of DataType A list containing the data type of each column in order. See Also -------- schema Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6.0, 7.0, 8.0], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.dtypes [Int64, Float64, String] >>> df shape: (3, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6.0 ┆ a │ │ 2 ┆ 7.0 ┆ b │ │ 3 ┆ 8.0 ┆ c │ └─────┴─────┴─────┘ )rdtypesr s rrDataFrame.dtypeswsNxx  rc^URVs0sHoXR_M sn$s snf)z Get flags that are set on the columns of this DataFrame. Returns ------- dict Mapping from column names to column flags. )rflagsrnames rrDataFrame.flagss+48<<@<4dj&&&<@@@s*cR[[URUR5SS9$)a* Get an ordered mapping of column names to their data type. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6.0, 7.0, 8.0], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.schema Schema({'foo': Int64, 'bar': Float64, 'ham': String}) F) check_dtypes)rdziprrr s rrDataFrame.schemas "c$,, 45IIrc UcSup4O%USLaSup4OUSLaSup4OSU<3n[U5eURX4S9nUbCXR:wa4USLaSURS US 3n[U5eUR U5nU$) a7 Return a NumPy ndarray with the given data type. This method ensures a Polars DataFrame can be treated as a NumPy ndarray. It enables `np.asarray` and NumPy universal functions. See the NumPy documentation for more information: https://numpy.org/doc/stable/user/basics.interoperability.html#the-array-method FTT)TTF)FFzinvalid input for `copy`: writable allow_copyzcopy not allowed: cast from z to z prohibited)rto_numpydtype RuntimeError __array__)rr+copyr(r)rarrs rr-DataFrame.__array__s <#. Hj T\#- Hj U]#/ Hj.th7CC. mmXmE  ))!3u}4SYYKtE7+V"3''--&C rcBU(a Sn[U5eSSKJn U"XS9$)a Convert to a dataframe object implementing the dataframe interchange protocol. Parameters ---------- nan_as_null Overwrite null values in the data with `NaN`. .. warning:: This functionality has not been implemented and the parameter will be removed in a future version. Setting this to `True` will raise a `NotImplementedError`. allow_copy Allow memory to be copied to perform the conversion. If set to `False`, causes conversions that are not zero-copy to fail. Notes ----- Details on the Python dataframe interchange protocol: https://data-apis.org/dataframe-protocol/latest/index.html Examples -------- Convert a Polars DataFrame to a generic dataframe object and access some properties. >>> df = pl.DataFrame({"a": [1, 2], "b": [3.0, 4.0], "c": ["x", "y"]}) >>> dfi = df.__dataframe__() >>> dfi.num_rows() 2 >>> dfi.get_column(1).dtype (, 64, 'g', '=') zfunctionality for `nan_as_null` has not been implemented and the parameter will be removed in a future version Use the default `nan_as_null=False`.rr)r))NotImplementedErrorpolars.interchange.dataframer)r nan_as_nullr)rrs r __dataframe__DataFrame.__dataframe__s-L ;  &c* *@t;;rcp[U[5(aURX5$URX5$)z(Compare a DataFrame with another object.)rr_compare_to_other_df_compare_to_non_df)rotherops r_compDataFrame._comps1 eY ' ',,U7 7**55 5rcURUR:wa Sn[U5eURUR:wa Sn[U5eSnUR[R "5R RU55n[R"X/SS9nUS:XaMURVs/sH4n[R"U5[R"UU35:HPM6 nnGOUS:XaMURVs/sH4n[R"U5[R"UU35:gPM6 nnGOZUS:XaMURVs/sH4n[R"U5[R"UU35:PM6 nnGOUS :XaLURVs/sH4n[R"U5[R"UU35:PM6 nnOUS :XaLURVs/sH4n[R"U5[R"UU35:PM6 nnOcUS :XaLURVs/sH4n[R"U5[R"UU35:*PM6 nnOS U<3n[U5eURU5$s snfs snfs snfs snfs snfs snf) z+Compare a DataFrame with another DataFrame.zDataFrame columns do not matchz!DataFrame dimensions do not match__POLARS_CMP_OTHER horizontalhoweqneqgtltgt_eqlt_equnexpected comparison operator ) rrr selectFallrsuffixconcatra) rr:r;rrM other_renamedcombinednexprs rr8DataFrame._compare_to_other_dfs1 <<5== (2CS/ ! :: $5CS/ !% QUUW\\%8%8%@A 88T1|D :?C||L|!AEE!HF8n 55|DLD 5[?C||L|!AEE!HF8n 55|DLD 4Z>BllKlAEE!Hquus6(^44lDKD 4Z>BllKlAEE!Hquus6(^44lDKD 7]?C||L|!AEE!HF8n 55|DLD 7]?C||L|!AEE!HF8n 55|DLD3B6:CS/ !t$$MLKKLLs$8;J5 ;J:;J?1;K;K ;KcV[U5 US:Xa'UR[R"5U:H5$US:Xa'UR[R"5U:g5$US:Xa'UR[R"5U:5$US:Xa'UR[R"5U:5$US:Xa'UR[R"5U:5$US:Xa'UR[R"5U:*5$SU<3n[ U5e)z0Compare a DataFrame with a non-DataFrame object.rCrDrErFrGrHrI)r6rJrKrLr)rr:r;rs rr9DataFrame._compare_to_non_df=s U# :;;quuw%/0 0 5[;;quuw%/0 0 4Z;;quuw/ / 4Z;;quuw/ / 7];;quuw%/0 0 7];;quuw%/0 03B6:CS/ !rc [U[R5(agU(a0UR[R "5[ U5-5$UR[R "5[ U5- 5$[U[5(dS[XRS9n[[UR5Vs/sHoCRSU35PM sn5nURnURU[[ 5nUR#UR$R'UR$55nU(dUOOUR)UVs/sH3o3R*R-5(dM#UR/5PM5 sn5nU(a[1UR2R555VVVs/sH^unupxUR75(dMXTR75(dXT[8:XdMD[;U5R=U5PM` n nnnU (aUR)U 5$U$s snfs snfs snnnf)N)lengthrQ)rrrurJrKrLrbr_prepare_other_argrrangeraliasr_cast_all_from_torLrBrrdiv_df with_columnsr+is_floatfloor enumerateritems is_integerrErar) rr:floordivsi orig_dtypesrrtp int_castss r_divDataFrame._divTs eRYY ' '{{1557c%j#899;;quuwU34 4E9--"5=Atzz9JK9JAww1#w/9JKLEll &&ungF __TXX__UYY7 8 R!NR77;K;K;M)!'')R!NO (11B1B1D'E'EOA|==?%!^..00KNd4J%F   $'E  y11 +L"Os$ I""III3!IIcUVs/sH>oDRU;dMURU5RUR5PM@ nnU(aUR U5$U$s snfr)r+rrZrr])rrfrom_tordcastss rr[DataFrame._cast_all_from_tossW46J2aE9I)!!!&&)2J).ru%6B6Ks A&.A&c"URUSS9$)NTrcrirr:s r __floordiv__DataFrame.__floordiv__ysyyy..rc"URUSS9$)NFrqrrrss r __truediv__DataFrame.__truediv__|syyy//rcSn[U5e)Nzqthe truth value of a DataFrame is ambiguous Hint: to check if a DataFrame contains any values, use `is_empty()`.)rrs r__bool__DataFrame.__bool__s W nrc&URUS5$)NrCr<rss r__eq__DataFrame.__eq__zz%&&rc&URUS5$)NrDr}rss r__ne__DataFrame.__ne__szz%''rc&URUS5$)NrEr}rss r__gt__DataFrame.__gt__rrc&URUS5$)NrFr}rss r__lt__DataFrame.__lt__rrc&URUS5$)NrGr}rss r__ge__DataFrame.__ge__zz%))rc&URUS5$)NrHr}rss r__le__DataFrame.__le__rrc"UR5$r) serializer s r __getstate__DataFrame.__getstate__s~~rcVUR[U55RUlgr)rr r)rstates r __setstate__DataFrame.__setstate__s##GEN377rc[U[5(a4URURR UR55$[ U5nURURR UR55$r)rrrrmul_dfrXmulrrss r__mul__DataFrame.__mul__sY eY ' '??488??599#=> >"5)txx||EHH566rc X-$rrss r__rmul__DataFrame.__rmul__s |rc[U[5(a4URURR UR55$[ U5nURURR UR55$r)rrrradd_dfrXaddrrss r__add__DataFrame.__add__s[ eY ' '??488??599#=> >"5)txx||EHH566rc[U[5(aIUR[U5[R "S5-R R55$X-$)N*)rrrJrbrKrarkeeprss r__radd__DataFrame.__radd__sH eS ! !;;E QUU3Z 7==BBDE E|rc[U[5(a4URURR UR55$[ U5nURURR UR55$r)rrrrsub_dfrXsubrrss r__sub__DataFrame.__sub__Y eY ' '??488??599#=> >"5)txx||EHH566rc[U[5(a4URURR UR55$[ U5nURURR UR55$r)rrrrrem_dfrXremrrss r__mod__DataFrame.__mod__rrc6URR5$r)ras_strr s r__str__DataFrame.__str__sxx  rc"UR5$r)rr s r__repr__DataFrame.__repr__s||~rcXR;$rrrkeys r __contains__DataFrame.__contains__sll""rc"UR5$r) iter_columnsr s r__iter__DataFrame.__iter__s  ""rc4[UR55$r)reversed get_columnsr s r __reversed__DataFrame.__reversed__s((*++rcgrrrs r __getitem__DataFrame.__getitem__srcgrrrs rrrsrcgrrrs rrrrc[X5$)u Get part of the DataFrame as a new DataFrame, Series, or scalar. Parameters ---------- key Rows / columns to select. This is easiest to explain via example. Suppose we have a DataFrame with columns `'a'`, `'d'`, `'c'`, `'d'`. Here is what various types of `key` would do: - `df[0, 'a']` extracts the first element of column `'a'` and returns a scalar. - `df[0]` extracts the first row and returns a Dataframe. - `df['a']` extracts column `'a'` and returns a Series. - `df[0:2]` extracts the first two rows and returns a Dataframe. - `df[0:2, 'a']` extracts the first two rows from column `'a'` and returns a Series. - `df[0:2, 0]` extracts the first two rows from the first column and returns a Series. - `df[[0, 1], [0, 1, 2]]` extracts the first two rows and the first three columns and returns a Dataframe. - `df[0: 2, ['a', 'c']]` extracts the first two rows from columns `'a'` and `'c'` and returns a Dataframe. - `df[:, 0: 2]` extracts all rows from the first two columns and returns a Dataframe. - `df[:, 'a': 'c']` extracts all rows and all columns positioned between `'a'` and `'c'` *inclusive* and returns a Dataframe. In our example, that would extract columns `'a'`, `'d'`, and `'c'`. Returns ------- DataFrame, Series, or scalar, depending on `key`. Examples -------- >>> df = pl.DataFrame( ... {"a": [1, 2, 3], "d": [4, 5, 6], "c": [1, 3, 2], "b": [7, 8, 9]} ... ) >>> df[0] shape: (1, 4) ┌─────┬─────┬─────┬─────┐ │ a ┆ d ┆ c ┆ b │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╪═════╡ │ 1 ┆ 4 ┆ 1 ┆ 7 │ └─────┴─────┴─────┴─────┘ >>> df[0, "a"] 1 >>> df["a"] shape: (3,) Series: 'a' [i64] [ 1 2 3 ] >>> df[0:2] shape: (2, 4) ┌─────┬─────┬─────┬─────┐ │ a ┆ d ┆ c ┆ b │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╪═════╡ │ 1 ┆ 4 ┆ 1 ┆ 7 │ │ 2 ┆ 5 ┆ 3 ┆ 8 │ └─────┴─────┴─────┴─────┘ >>> df[0:2, "a"] shape: (2,) Series: 'a' [i64] [ 1 2 ] >>> df[0:2, 0] shape: (2,) Series: 'a' [i64] [ 1 2 ] >>> df[[0, 1], [0, 1, 2]] shape: (2, 3) ┌─────┬─────┬─────┐ │ a ┆ d ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ 1 ┆ 4 ┆ 1 │ │ 2 ┆ 5 ┆ 3 │ └─────┴─────┴─────┘ >>> df[0:2, ["a", "c"]] shape: (2, 2) ┌─────┬─────┐ │ a ┆ c │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 1 │ │ 2 ┆ 3 │ └─────┴─────┘ >>> df[:, 0:2] shape: (3, 2) ┌─────┬─────┐ │ a ┆ d │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 4 │ │ 2 ┆ 5 │ │ 3 ┆ 6 │ └─────┴─────┘ >>> df[:, "a":"c"] shape: (3, 3) ┌─────┬─────┬─────┐ │ a ┆ d ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ 1 ┆ 4 ┆ 1 │ │ 2 ┆ 5 ┆ 3 │ │ 3 ┆ 6 ┆ 2 │ └─────┴─────┴─────┘ r'rs rrrsR"$,,rc [U[5(a Sn[U5e[U[5(a[R "U5nUR S:wa Sn[U5eURS[U5:wa Sn[U5e/n[U5H1upVUR[R"XbSS2U455 M3 URU5RUlg[U[ 5(aUupx[U[R5(aUR"[$:Xd['U5(a Sn[U5e[U[5(aUR)U5n O0[U[*5(a USS2U4n OSU<3n[U5eX)U'[U[*5(aUR-X5 g[U[5(aUR/X5 ggS U<S [1U5R2<S U<S [1U5R2<3n[U5e) ug Modify DataFrame elements in place, using assignment syntax. Parameters ---------- key : str | Sequence[int] | Sequence[str] | tuple[Any, str | int] Specifies the location(s) within the DataFrame to assign new values. The behavior varies based on the type of `key`: - Str: `df["a"] = value`: Not supported. Raises a `TypeError`. Use `df.with_columns(...)` to add or modify columns. - Sequence[str]: `df[["a", "b"]] = value`: Assigns multiple columns at once. `value` must be a 2D array-like structure with the same number of columns as the list of column names provided. - tuple[Any, str | int]: `df[row_idx, "a"] = value`: Assigns a new value to a specific element in the DataFrame, where `row_idx` specifies the row and `"a"` specifies the column. - `df[row_idx, col_idx] = value`: Similar to the above, but `col_idx` is the integer index of the column. value : Any The new value(s) to assign. The expected structure of `value` depends on the form of `key`: - For multiple column assignment (`df[["a", "b"]] = value`), `value` should be a 2D array-like object with shape (n_rows, n_columns). - For single element assignment (`df[row_idx, "a"] = value`), `value` should be a scalar. Raises ------ TypeError If an unsupported assignment is attempted, such as assigning a Series directly to a column using `df["a"] = series`. ValueError If the shape of `value` does not match the expected shape based on `key`. Examples -------- Sequence[str] : `df[["a", "b"]] = value`: >>> import numpy as np >>> df = pl.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]}) >>> df[["a", "b"]] = np.array([[10, 40], [20, 50], [30, 60]]) >>> df shape: (3, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 10 ┆ 40 │ │ 20 ┆ 50 │ │ 30 ┆ 60 │ └─────┴─────┘ tuple[Any, str | int] : `df[row_idx, "a"] = value`: >>> df[1, "a"] = 100 >>> df shape: (3, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 10 ┆ 40 │ │ 100 ┆ 50 │ │ 30 ┆ 60 │ └─────┴─────┘ `df[row_idx, col_idx] = value`: >>> df[0, 1] = 30 >>> df shape: (3, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 10 ┆ 30 │ │ 100 ┆ 50 │ │ 30 ┆ 60 │ └─────┴─────┘ z]DataFrame object does not support `Series` assignment by index Use `DataFrame.with_columns`.z,can only set multiple columns with 2D matrixzEmatrix columns should be equal to list used to determine column namesNzjnot allowed to set DataFrame by boolean mask in the row position Consider using `DataFrame.with_columns`.zunexpected column selection z/cannot use `__setitem__` on DataFrame with key z of type z and value )rrrrrarrayndimrr lenr`appendrrur]rrr+r@r/rintreplace_columnrrr) rrvaluerrrer row_selection col_selectionrds r __setitem__DataFrame.__setitem__usF c3  4 C. T " "HHUOEzzQD o%{{1~S)] o%G$S>ryyQT{;<*((155DHU # #+. (M="))449L9LPW9W!-00C n$---$$]3M3//M)*4]4EFn$ %m ---##M5M3// m/0 G9T#Y-?-?,BeYiU 0D0D/GI  C. rcUR$r)rr s r__len__DataFrame.__len__s {{rc"UR5$rcloner s r__copy__DataFrame.__copy__zz|rc"UR5$rr)rmemos r __deepcopy__DataFrame.__deepcopy__rrcUR$rrr s r_ipython_key_completions_#DataFrame._ipython_key_completions_!s ||rc8URRU5$)z Export a DataFrame via the Arrow PyCapsule Interface. https://arrow.apache.org/docs/dev/format/CDataInterface/PyCapsuleInterface.html )rr)rrequested_schemas rrDataFrame.__arrow_c_stream__$s xx**+;<>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6.0, 7.0, 8.0], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.collect_schema() Schema({'foo': Int64, 'bar': Float64, 'ham': String}) Access various properties of the schema using the :class:`Schema` object. >>> schema = df.collect_schema() >>> schema["bar"] Float64 >>> schema.names() ['foo', 'bar', 'ham'] >>> schema.dtypes() [Int64, Float64, String] >>> schema.len() 3 rr s rcollect_schemaDataFrame.collect_schemaGsR{{rcUcXUcUURS:waSUR<3n[U5eURRS5R S5$UbUc Sn[U5e[ U[ 5(aURRU5OURRU5nURU5$)aY Return the DataFrame as a scalar, or return the element at the given row/column. Parameters ---------- row Optional row index. column Optional column index or name. See Also -------- row : Get the values of a single row, either by index or by predicate. Notes ----- If row/col not provided, this is equivalent to `df[0,0]`, with a check that the shape is (1,1). With row/col, this is equivalent to `df[row,col]`. Examples -------- >>> df = pl.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]}) >>> df.select((pl.col("a") * pl.col("b")).sum()).item() 32 >>> df.item(1, 1) 5 >>> df.item(2, "b") 6 )rrzycan only call `.item()` if the dataframe is of shape (1, 1), or if explicit row/col values are provided; frame has shape rz8cannot call `.item()` with only one of `row` or `column`) r rr to_series get_indexrr get_columnget_index_signed)rrowrrrds ritemDataFrame.itemrs< ;6>zzV#((, ~7 !o%88%%a(2215 5 [FNLCS/ !&#&& HH  v &$$V, !!#&&rfuture compat_levelz1.1versionrcUR(d[R"05$UcSnO![U[5(a UR nUR RW5n[RRU5$)aP Collect the underlying arrow arrays in an Arrow Table. This operation is mostly zero copy. Data types that do copy: - CategoricalType .. versionchanged:: 1.1 The `future` parameter was renamed `compat_level`. Parameters ---------- compat_level Use a specific compatibility level when exporting Polars' internal data structures. Examples -------- >>> df = pl.DataFrame( ... {"foo": [1, 2, 3, 4, 5, 6], "bar": ["a", "b", "c", "d", "e", "f"]} ... ) >>> df.to_arrow() pyarrow.Table foo: int64 bar: large_string ---- foo: [[1,2,3,4,5,6]] bar: [["a","b","c","d","e","f"]] F) rrtablerrc_versionrto_arrowr from_batches)rrcompat_level_pyrecord_batchess rrDataFrame.to_arrowsh@zz88B<   #O  k 2 2*33O**?;xx$$^44r.) as_seriescgrrrr s rto_dictDataFrame.to_dictsORrcgrrr"s rr#r$MPrcgrrr"s rr#r$s47rcU(aUVs0sHo"RU_M sn$UVs0sHo"RUR5_M sn$s snfs snf)u* Convert DataFrame to a dictionary mapping column name to values. Parameters ---------- as_series True -> Values are Series False -> Values are List[Any] See Also -------- rows_by_key to_dicts Examples -------- >>> df = pl.DataFrame( ... { ... "A": [1, 2, 3, 4, 5], ... "fruits": ["banana", "banana", "apple", "apple", "banana"], ... "B": [5, 4, 3, 2, 1], ... "cars": ["beetle", "audi", "beetle", "beetle", "beetle"], ... "optional": [28, 300, None, 2, -30], ... } ... ) >>> df shape: (5, 5) ┌─────┬────────┬─────┬────────┬──────────┐ │ A ┆ fruits ┆ B ┆ cars ┆ optional │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ str ┆ i64 ┆ str ┆ i64 │ ╞═════╪════════╪═════╪════════╪══════════╡ │ 1 ┆ banana ┆ 5 ┆ beetle ┆ 28 │ │ 2 ┆ banana ┆ 4 ┆ audi ┆ 300 │ │ 3 ┆ apple ┆ 3 ┆ beetle ┆ null │ │ 4 ┆ apple ┆ 2 ┆ beetle ┆ 2 │ │ 5 ┆ banana ┆ 1 ┆ beetle ┆ -30 │ └─────┴────────┴─────┴────────┴──────────┘ >>> df.to_dict(as_series=False) {'A': [1, 2, 3, 4, 5], 'fruits': ['banana', 'banana', 'apple', 'apple', 'banana'], 'B': [5, 4, 3, 2, 1], 'cars': ['beetle', 'audi', 'beetle', 'beetle', 'beetle'], 'optional': [28, 300, None, 2, -30]} >>> df.to_dict(as_series=True) {'A': shape: (5,) Series: 'A' [i64] [ 1 2 3 4 5 ], 'fruits': shape: (5,) Series: 'fruits' [str] [ "banana" "banana" "apple" "apple" "banana" ], 'B': shape: (5,) Series: 'B' [i64] [ 5 4 3 2 1 ], 'cars': shape: (5,) Series: 'cars' [str] [ "beetle" "audi" "beetle" "beetle" "beetle" ], 'optional': shape: (5,) Series: 'optional' [i64] [ 28 300 null 2 -30 ]} )rto_list)rr rds rr#r$sMt '+,t!FFAIt, ,156AFFAIIK'6 6-6s A$Ac URSS9$)u Convert every row to a dictionary of Python-native values. Notes ----- If you have `ns`-precision temporal values you should be aware that Python natively only supports up to `μs`-precision; `ns`-precision values will be truncated to microseconds on conversion to Python. If this matters to your use-case you should export to a different format (such as Arrow or NumPy). Examples -------- >>> df = pl.DataFrame({"foo": [1, 2, 3], "bar": [4, 5, 6]}) >>> df.to_dicts() [{'foo': 1, 'bar': 4}, {'foo': 2, 'bar': 5}, {'foo': 3, 'bar': 6}] Tnamed)rowsr s rto_dictsDataFrame.to_dicts;s"yyty$$rfortran)orderr(r) structured use_pyarrowcUb [SSS9 U(GaYU(d"UR5(d Sn[U5e/n/nUR5Hn U R[ :Xa*U R R5RSSUS9n OU RUS9n U R[:Xa)U R5(dU R[S S 9n URU 5 URU RU RU RS S45 M [ R""UR$US 9n ['UR(5H upX|X'M U $UR*RXUS 9$)a Convert this DataFrame to a NumPy ndarray. This operation copies data only when necessary. The conversion is zero copy when all of the following hold: - The DataFrame is fully contiguous in memory, with all Series back-to-back and all Series consisting of a single chunk. - The data type is an integer or float. - The DataFrame contains no null values. - The `order` parameter is set to `fortran` (default). - The `writable` parameter is set to `False` (default). Parameters ---------- order The index order of the returned NumPy array, either C-like or Fortran-like. In general, using the Fortran-like index order is faster. However, the C-like order might be more appropriate to use for downstream applications to prevent cloning data, e.g. when reshaping into a one-dimensional array. writable Ensure the resulting array is writable. This will force a copy of the data if the array was created without copy, as the underlying Arrow data is immutable. allow_copy Allow memory to be copied to perform the conversion. If set to `False`, causes conversions that are not zero-copy to fail. structured Return a `structured array`_ with a data type that corresponds to the DataFrame schema. If set to `False` (default), a 2D ndarray is returned instead. .. _structured array: https://numpy.org/doc/stable/user/basics.rec.html use_pyarrow Use `pyarrow.Array.to_numpy `_ function for the conversion to NumPy if necessary. .. deprecated:: 0.20.28 Polars now uses its native engine by default for conversion to NumPy. Examples -------- Numeric data without nulls can be converted without copying data in some cases. The resulting array will not be writable. >>> df = pl.DataFrame({"a": [1, 2, 3]}) >>> arr = df.to_numpy() >>> arr array([[1], [2], [3]]) >>> arr.flags.writeable False Set `writable=True` to force data copy to make the array writable. >>> df.to_numpy(writable=True).flags.writeable True If the DataFrame contains different numeric data types, the resulting data type will be the supertype. This requires data to be copied. Integer types with nulls are cast to a float type with `nan` representing a null value. >>> df = pl.DataFrame({"a": [1, 2, None], "b": [4.0, 5.0, 6.0]}) >>> df.to_numpy() array([[ 1., 4.], [ 2., 5.], [nan, 6.]]) Set `allow_copy=False` to raise an error if data would be copied. >>> s.to_numpy(allow_copy=False) # doctest: +SKIP Traceback (most recent call last): ... RuntimeError: copy not allowed: cannot convert to a NumPy array without copying data Polars defaults to F-contiguous order. Use `order="c"` to force the resulting array to be C-contiguous. >>> df.to_numpy(order="c").flags.c_contiguous True DataFrames with mixed types will result in an array with an object dtype. >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6.5, 7.0, 8.5], ... "ham": ["a", "b", "c"], ... }, ... schema_overrides={"foo": pl.UInt8, "bar": pl.Float32}, ... ) >>> df.to_numpy() array([[1, 6.5, 'a'], [2, 7.0, 'b'], [3, 8.5, 'c']], dtype=object) Set `structured=True` to convert to a structured array, which can better preserve individual column data such as name and data type. >>> df.to_numpy(structured=True) array([(1, 6.5, 'a'), (2, 7. , 'b'), (3, 8.5, 'c')], dtype=[('foo', 'u1'), ('bar', '>> df = pl.DataFrame( ... { ... "lbl": [0, 1, 2, 3], ... "feat1": [1, 0, 0, 1], ... "feat2": [1.5, -0.5, 0.0, -2.25], ... } ... ) Standard return type (2D Array), on the standard device: >>> df.to_jax() Array([[ 0. , 1. , 1.5 ], [ 1. , 0. , -0.5 ], [ 2. , 0. , 0. ], [ 3. , 1. , -2.25]], dtype=float32) Create the Array on the default GPU device: >>> a = df.to_jax(device="gpu") # doctest: +SKIP >>> a.device() # doctest: +SKIP GpuDevice(id=0, process_index=0) Create the Array on a specific GPU device: >>> gpu_device = jax.devices("gpu")[1] # doctest: +SKIP >>> a = df.to_jax(device=gpu_device) # doctest: +SKIP >>> a.device() # doctest: +SKIP GpuDevice(id=1, process_index=0) As a dictionary of individual Arrays: >>> df.to_jax("dict") {'lbl': Array([0, 1, 2, 3], dtype=int32), 'feat1': Array([1, 0, 0, 1], dtype=int32), 'feat2': Array([ 1.5 , -0.5 , 0. , -2.25], dtype=float32)} As a "label" and "features" dictionary; note that as "features" is not declared, it defaults to all the columns that are not in "label": >>> df.to_jax("dict", label="lbl") {'label': Array([[0], [1], [2], [3]], dtype=int32), 'features': Array([[ 1. , 1.5 ], [ 0. , -0.5 ], [ 0. , 0. ], [ 1. , -2.25]], dtype=float32)} As a "label" and "features" dictionary where each is designated using a col or selector expression (which can also be used to cast the data if the label and features are better-represented with different dtypes): >>> import polars.selectors as cs >>> df.to_jax( ... return_type="dict", ... features=cs.float(), ... label=pl.col("lbl").cast(pl.UInt8), ... ) {'label': Array([[0], [1], [2], [3]], dtype=uint8), 'features': Array([[ 1.5 ], [-0.5 ], [ 0. ], [-2.25]], dtype=float32)} rNz>`label` and `features` only apply when `return_type` is 'dict'B`label` is required if setting `features` when `return_type='dict'jaxzPlease see `https://jax.readthedocs.io/en/latest/installation.html` for specific installation recommendations for the Jax package)install_messageJAX_ENABLE_X640rrframe_to_numpyFz Jax Array)rr1r(targetK)ar1rBrC, invalid `return_type`:  Expected one of: )"rrWconfigjax_enable_x64boolrrrrrrBrArDrCrKrJrrdevices contextlib nullcontextdefault_devicepolars.ml.utilitiesrQrYasarrayrJdroprrGrrrr)rrFrArBrCr+r1rjxenabled_double_precisionframerQr/ label_framefeatures_framesrsvalid_jax_typess rrGrHs+x & e&78;ORCS/ ! F "u}9MVCS/ !  L  $&99#;#;$ t  /5 6@   IIe$E)IIwvvNOEE fc " "ZZ'*F)/Z # # %R=N=Nv=V Vg%>$"&  xx''#S'9W V&$"',,u"5K$/ X."ZZ)<)<=# "-!3!3!5$2$9$9$;-W V8?DDesHHcjjl2eD9W V<#'))H],C"D/ >QRaQbc o% E9W Vs+0/I)A'II$II7I I)rBrCr+cgrrrrFrBrCr+s rto_torchDataFrame.to_torchsrcgrrrks rrlrmsrcgrrrks rrlrms#&rc<US;aUcUb Sn[U5eUS:XaUcUb Sn[U5e[S5nU[[[4;aSU3n[U5eU=(d [[ [[ [[ 0nUbURU5n[U[R5(dURU5nUbURU5OUR"UR6RU5n [R"X/SS 9n O%UbURU5OURU5n US :Xa S S KJn U "U S SS9n UR$"U 5$US:XaOUb!WR'5W R'5S.$U V s0sHoR(U R'5_M sn $US:XaS SKJn UcSO WRnU"XUS9$SR/[1[255nSU<SU3n[U5es sn f)a Convert DataFrame to a PyTorch Tensor, Dataset, or dict of Tensors. .. versionadded:: 0.20.23 .. warning:: This functionality is currently considered **unstable**. It may be changed at any point without it being considered a breaking change. Parameters ---------- return_type : {"tensor", "dataset", "dict"} Set return type; a PyTorch Tensor, PolarsDataset (a frame-specialized TensorDataset), or dict of Tensors. label One or more column names, expressions, or selectors that label the feature data; when `return_type` is "dataset", the PolarsDataset will return `(features, label)` tensor tuples for each row. Otherwise, it returns `(features,)` tensor tuples where the feature contains all the row data. features One or more column names, expressions, or selectors that contain the feature data; if omitted, all columns that are not designated as part of the label are used. dtype Unify the dtype of all returned tensors; this casts any column that is not of the required dtype before converting to Tensor. This includes the label column *unless* the label is an expression (such as `pl.col("label_column").cast(pl.Int16)`). See Also -------- to_dummies to_jax to_numpy Examples -------- >>> df = pl.DataFrame( ... { ... "lbl": [0, 1, 2, 3], ... "feat1": [1, 0, 0, 1], ... "feat2": [1.5, -0.5, 0.0, -2.25], ... } ... ) Standard return type (Tensor), with f32 supertype: >>> df.to_torch(dtype=pl.Float32) tensor([[ 0.0000, 1.0000, 1.5000], [ 1.0000, 0.0000, -0.5000], [ 2.0000, 0.0000, 0.0000], [ 3.0000, 1.0000, -2.2500]]) As a dictionary of individual Tensors: >>> df.to_torch("dict") {'lbl': tensor([0, 1, 2, 3]), 'feat1': tensor([1, 0, 0, 1]), 'feat2': tensor([ 1.5000, -0.5000, 0.0000, -2.2500], dtype=torch.float64)} As a "label" and "features" dictionary; note that as "features" is not declared, it defaults to all the columns that are not in "label": >>> df.to_torch("dict", label="lbl", dtype=pl.Float32) {'label': tensor([[0.], [1.], [2.], [3.]]), 'features': tensor([[ 1.0000, 1.5000], [ 0.0000, -0.5000], [ 0.0000, 0.0000], [ 1.0000, -2.2500]])} As a PolarsDataset, with f64 supertype: >>> ds = df.to_torch("dataset", dtype=pl.Float64) >>> ds[3] (tensor([ 3.0000, 1.0000, -2.2500], dtype=torch.float64),) >>> ds[:2] (tensor([[ 0.0000, 1.0000, 1.5000], [ 1.0000, 0.0000, -0.5000]], dtype=torch.float64),) >>> ds[[0, 3]] (tensor([[ 0.0000, 1.0000, 1.5000], [ 3.0000, 1.0000, -2.2500]], dtype=torch.float64),) As a convenience the PolarsDataset can opt in to half-precision data for experimentation (usually this would be set on the model/pipeline): >>> list(ds.half()) [(tensor([0.0000, 1.0000, 1.5000], dtype=torch.float16),), (tensor([ 1.0000, 0.0000, -0.5000], dtype=torch.float16),), (tensor([2., 0., 0.], dtype=torch.float16),), (tensor([ 3.0000, 1.0000, -2.2500], dtype=torch.float16),)] Pass PolarsDataset to a DataLoader, designating the label: >>> from torch.utils.data import DataLoader >>> ds = df.to_torch("dataset", label="lbl") >>> dl = DataLoader(ds, batch_size=2) >>> batches = list(dl) >>> batches[0] [tensor([[ 1.0000, 1.5000], [ 0.0000, -0.5000]], dtype=torch.float64), tensor([0, 1])] Note that labels can be given as expressions, allowing them to have a dtype independent of the feature columns (multi-column labels are supported). >>> ds = df.to_torch( ... return_type="dataset", ... dtype=pl.Float32, ... label=pl.col("lbl").cast(pl.Int16), ... ) >>> ds[:2] (tensor([[ 1.0000, 1.5000], [ 0.0000, -0.5000]]), tensor([0, 1], dtype=torch.int16)) Easily integrate with (for example) scikit-learn and other datasets: >>> from sklearn.datasets import fetch_california_housing # doctest: +SKIP >>> housing = fetch_california_housing() # doctest: +SKIP >>> df = pl.DataFrame( ... data=housing.data, ... schema=housing.feature_names, ... ).with_columns( ... Target=housing.target, ... ) # doctest: +SKIP >>> train = df.to_torch("dataset", label="Target") # doctest: +SKIP >>> loader = DataLoader( ... train, ... shuffle=True, ... batch_size=64, ... ) # doctest: +SKIP )datasetrNzK`label` and `features` only apply when `return_type` is 'dataset' or 'dict'rrKrXz8PyTorch does not support u16, u32, or u64 dtypes; given r@rAtensorrrPTr)r(rRrUrqrrVrWrX)rrWrIrJrKrCrDrJrrrsrrbrrKrNr`rQ from_numpyrlrpolars.ml.torchrrrr)rrFrBrCr+rrXto_dtyperfrgrerQr/rhr pds_labelvalid_torch_typess rrlrms^ 1 1  !5_CS/ ! F "u}9MVCS/ !( VVV, ,LUGTCS/ !IVUFE65I  ++e,KeRWW--)..x8 ' H%YY 3 34d8n  HHk: ME.6.BT[[*RRE ( " : hGC##C( ( F " )113 . 7 7 9 =BBES#,,.0EBB I % 5 % ;3F3FI (K K $ (?*C D +K?:MN_M`aCS/ !Cs$Huse_pyarrow_extension_arrayc U(a[[R5S:aS[R<3n[U5e[(a[[ R5S:a;Sn[(a#US[ R<S3- n[U5e[ U5e[UR;aUR"SSU0UD6$UR"U4SU0UD6$) a Convert this DataFrame to a pandas DataFrame. This operation copies data if `use_pyarrow_extension_array` is not enabled. Parameters ---------- use_pyarrow_extension_array Use PyArrow-backed extension arrays instead of NumPy arrays for the columns of the pandas DataFrame. This allows zero copy operations and preservation of null values. Subsequent operations on the resulting pandas DataFrame may trigger conversion to NumPy if those operations are not supported by PyArrow compute functions. **kwargs Additional keyword arguments to be passed to :meth:`pyarrow.Table.to_pandas`. Returns ------- :class:`pandas.DataFrame` Notes ----- This operation requires that both :mod:`pandas` and :mod:`pyarrow` are installed. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6.0, 7.0, 8.0], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.to_pandas() foo bar ham 0 1 6.0 a 1 2 7.0 b 2 3 8.0 c Null values in numeric columns are converted to `NaN`. >>> df = pl.DataFrame( ... { ... "foo": [1, 2, None], ... "bar": [6.0, None, 8.0], ... "ham": [None, "b", "c"], ... } ... ) >>> df.to_pandas() foo bar ham 0 1.0 6.0 None 1 2.0 NaN b 2 NaN 8.0 c Pass `use_pyarrow_extension_array=True` to get a pandas DataFrame with columns backed by PyArrow extension arrays. This will preserve null values. >>> df.to_pandas(use_pyarrow_extension_array=True) foo bar ham 0 1 6.0 1 2 b 2 8.0 c >>> _.dtypes foo int64[pyarrow] bar double[pyarrow] ham large_string[pyarrow] dtype: object rrz\pandas>=1.5.0 is required for `to_pandas("use_pyarrow_extension_array=True")`, found Pandas )rzLpyarrow>=8.0.0 is required for `to_pandas(use_pyarrow_extension_array=True)`z, found pyarrow .ryr) r2rrr^rPrrrFr_to_pandas_with_object_columns!_to_pandas_without_object_columns)rrykwargsrs r to_pandasDataFrame.to_pandas sX 'R^^,v5tuwvDvDuGH055%%r~~)F)Od%%-bnn-?qAAC4S99-c22 T[[ 66,GKQ 55  .I MS  rc /n/n[UR5H>upVUR5(aURU5 M-URU5 M@ U(aUSS2U4nUR"U4SU0UD6nO[ R "5nUHAnURUn URXYURU5R55 MC U$)Nry) r`r is_objectrrrrrinsertr r) rryrobject_columnsnot_object_columnsrer+df_without_objects pandas_dfrs rr~(DataFrame._to_pandas_with_object_columns s!$++.HA  %%a("))!, / !%a);&;!< >>",GI  I A<<?D   QdnnQ&7&A&A&C D rc HUR(d[R"5$URR 5n[ R RU5nU(aUR"SSSSS.UD6$URSS5nUR"SSU0UD6$)NTc.[R"U5$r)r ArrowDtype)pa_dtypes r=DataFrame._to_pandas_without_object_columns..: s bmmH.Er) self_destruct split_blocks types_mapperdate_as_objectFr) rrrrrrrrpop)rrryrrtblrs rr+DataFrame._to_pandas_without_object_columns* sxx<<> !))+hh##N3 &=="!E   $4e<}}ENEfEErcJ[URRU55$)a| Select column as Series at index location. Parameters ---------- index Location of selection. See Also -------- get_column Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.to_series(1) shape: (3,) Series: 'bar' [i64] [ 6 7 8 ] )r9rr )rindexs rr DataFrame.to_seriesA s>dhh((/00rcT[5nURS5 [UR5HTnURS5 URUR U5R U55 URS5 MV URS5 UR 5$)u; Convert DataFrame to instantiable string representation. Parameters ---------- n Only use first n rows. See Also -------- polars.Series.to_init_repr polars.from_repr Examples -------- >>> df = pl.DataFrame( ... [ ... pl.Series("foo", [1, 2, 3], dtype=pl.UInt8), ... pl.Series("bar", [6.0, 7.0, 8.0], dtype=pl.Float32), ... pl.Series("ham", ["a", "b", "c"], dtype=pl.String), ... ] ... ) >>> print(df.to_init_repr()) pl.DataFrame( [ pl.Series('foo', [1, 2, 3], dtype=pl.UInt8), pl.Series('bar', [6.0, 7.0, 8.0], dtype=pl.Float32), pl.Series('ham', ['a', 'b', 'c'], dtype=pl.String), ] ) >>> df_from_str_repr = eval(df.to_init_repr()) >>> df_from_str_repr shape: (3, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ u8 ┆ f32 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6.0 ┆ a │ │ 2 ┆ 7.0 ┆ b │ │ 3 ┆ 8.0 ┆ c │ └─────┴─────┴─────┘ zpl.DataFrame( [ z z, z ] ) )r writerYrr  to_init_reprr)rrQoutputres rrDataFrame.to_init_reprb sZ -.tzz"A LL $ LL*77: ; LL #  \"  rcgrrrfilers rrDataFrame.serialize srcgrrrs rrr sNQrcgrrrs rrr srcUS:XaURRnO.US:XaURRnOSU<3n[U5e[ X1U5$)u Serialize this DataFrame to a file or string in JSON format. Parameters ---------- file File path or writable file-like object to which the result will be written. If set to `None` (default), the output is returned as a string instead. format The format in which to serialize. Options: - `"binary"`: Serialize to binary format (bytes). This is the default. - `"json"`: Serialize to JSON format (string). Notes ----- Serialization is not stable across Polars versions: a LazyFrame serialized in one Polars version may not be deserializable in another Polars version. Examples -------- Serialize the DataFrame into a binary representation. >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... } ... ) >>> bytes = df.serialize() >>> type(bytes) The bytes can later be deserialized back into a DataFrame. >>> import io >>> pl.DataFrame.deserialize(io.BytesIO(bytes)) shape: (3, 2) ┌─────┬─────┐ │ foo ┆ bar │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 6 │ │ 2 ┆ 7 │ │ 3 ┆ 8 │ └─────┴─────┘ rrr)rserialize_binaryserialize_jsonrr,)rrr serializerrs rrr sVl X 22J v 00JFvjQCS/ !&z@@rcgrrrrs r write_jsonDataFrame.write_json s36rcgrrrs rrr =@rc@^SU4SjjnUcU"5$[U[5(aU"5nURU5 g[U[[45(a'[ U5nTR RU5 gTR RU5 g)a Serialize to JSON representation. Parameters ---------- file File path or writable file-like object to which the result will be written. If set to `None` (default), the output is returned as a string instead. See Also -------- DataFrame.write_ndjson Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... } ... ) >>> df.write_json() '[{"foo":1,"bar":6},{"foo":2,"bar":7},{"foo":3,"bar":8}]' c>[5nTRRU5 UR5nSSS5 WR S5$!,(df  N=f)Nutf8)r rrrdecode)buf json_bytesrs rwrite_json_to_string2DataFrame.write_json..write_json_to_string sGc##C( \\^ $$V, ,s ,A ANreturnr)rr rrr r1rr)rrrjson_strs` rrr s4 - <') ) h ' '+-H JJx  sDk * *%d+D HH   % HH   %rcgrrrs r write_ndjsonDataFrame.write_ndjson s69rcgrrrs rrr sLOrcJSnUc[S[55nSnO3[U[[R 45(a [ U5nOUnSnSSKJn UR5RUUR5US9 U(a[UR5S S 9$g) a Serialize to newline delimited JSON representation. Parameters ---------- file File path or writable file-like object to which the result will be written. If set to `None` (default), the output is returned as a string instead. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... } ... ) >>> df.write_ndjson() '{"foo":1,"bar":6}\n{"foo":2,"bar":7}\n{"foo":3,"bar":8}\n' FN IO[bytes]T in-memoryr QueryOptFlags) optimizationsengineutf-8encoding) rr rrrPathLiker1polars.lazyframe.opt_flagsrlazy sink_ndjson_eagerr)rrshould_return_bufferrRrrs rrr" s. % <+wy1F#' sBKK0 1 1'-FF(<  '..0  v(7; ;r) include_bominclude_header separatorline_terminator quote_char batch_sizedatetime_format date_format time_formatfloat_scientificfloat_precision decimal_comma null_value quote_stylestorage_optionscredential_providerretriescgrrrrrrrrrrrrrrrrrrrrrs r write_csvDataFrame.write_csvR s,rcgrrrs rrrj s,r, "iautorcSSKJn U"SUSS9 U"SUSS9 U(dSnSnUc[S [55nSnO3[ U[ [ R45(a [U5nOUnS nSS K J n UR5RUUUUUUUUU U U U U UUUUUUR5US 9 U(a[ UR5S S9$g)a$ Write to comma-separated values (CSV) file. Parameters ---------- file File path or writable file-like object to which the result will be written. If set to `None` (default), the output is returned as a string instead. include_bom Whether to include UTF-8 BOM in the CSV output. include_header Whether to include header in the CSV output. separator Separate CSV fields with this symbol. line_terminator String used to end each row. quote_char Byte to use as quoting character. batch_size Number of rows that will be processed per thread. datetime_format A format string, with the specifiers defined by the `chrono `_ Rust crate. If no format specified, the default fractional-second precision is inferred from the maximum timeunit found in the frame's Datetime cols (if any). date_format A format string, with the specifiers defined by the `chrono `_ Rust crate. time_format A format string, with the specifiers defined by the `chrono `_ Rust crate. float_scientific Whether to use scientific form always (true), never (false), or automatically (None) for `Float32` and `Float64` datatypes. float_precision Number of decimal places to write, applied to both `Float32` and `Float64` datatypes. decimal_comma Use a comma as the decimal separator instead of a point in standard notation. Floats will be encapsulated in quotes if necessary; set the field separator to override. null_value A string representing null values (defaulting to the empty string). quote_style : {'necessary', 'always', 'non_numeric', 'never'} Determines the quoting strategy used. - necessary (default): This puts quotes around fields only when necessary. They are necessary when fields contain a quote, separator or record terminator. Quotes are also necessary when writing an empty record (which is indistinguishable from a record with one empty field). This is the default. - always: This puts quotes around every field. Always. - never: This never puts quotes around fields, even if that results in invalid CSV data (e.g.: by not quoting strings containing the separator). - non_numeric: This puts quotes around all fields that are non-numeric. Namely, when writing a field that does not parse as a valid float or integer, then quotes will be used even if they aren`t strictly necessary. storage_options Options that indicate how to connect to a cloud provider. The cloud providers currently supported are AWS, GCP, and Azure. See supported keys here: * `aws `_ * `gcp `_ * `azure `_ * Hugging Face (`hf://`): Accepts an API key under the `token` parameter: `{'token': '...'}`, or by setting the `HF_TOKEN` environment variable. If `storage_options` is not provided, Polars will try to infer the information from environment variables. credential_provider Provide a function that can be called to provide cloud storage credentials. The function is expected to return a dictionary of credential keys along with an optional credential expiry time. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. retries Number of retries if accessing a cloud instance fails. Examples -------- >>> import pathlib >>> >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3, 4, 5], ... "bar": [6, 7, 8, 9, 10], ... "ham": ["a", "b", "c", "d", "e"], ... } ... ) >>> path: pathlib.Path = dirpath / "new_file.csv" >>> df.write_csv(path, separator=",") r)_check_arg_is_1byterF) can_be_emptyrTNrrr)rrrrrrrrrrrrrrrrrrrrr)polars.io.csv._utilsrrr rrrrr1rrrsink_csvrr)rrrrrrrrrrrrrrrrrrrrrrRrrs rrr sz =KGL*4HJ$ <+wy1F#' sBKK0 1 1'-FF(<  #)+!!+##-+'!#+ 3'..0)  . v(7; ;r )rc DUR"SSUS.UD6n[U5 g)a Copy `DataFrame` in csv format to the system clipboard with `write_csv`. Useful for pasting into Excel or other similar spreadsheet software. Parameters ---------- separator Separate CSV fields with this symbol. kwargs Additional arguments to pass to `write_csv`. See Also -------- polars.read_clipboard: Read a DataFrame from the clipboard. write_csv: Write to comma-separated values (CSV) file. N)rrr)r_write_clipboard_string)rrrresults rwrite_clipboardDataFrame.write_clipboard0 s$$nnN$)NvN'r uncompressedcUcSn[U[[45(a [U5nUcSnURR XU5 g)a; Write to Apache Avro file. Parameters ---------- file File path or writable file-like object to which the data will be written. compression : {'uncompressed', 'snappy', 'deflate'} Compression method. Defaults to "uncompressed". name Schema name. Defaults to empty string. Examples -------- >>> import pathlib >>> >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3, 4, 5], ... "bar": [6, 7, 8, 9, 10], ... "ham": ["a", "b", "c", "d", "e"], ... } ... ) >>> path: pathlib.Path = dirpath / "new_file.avro" >>> df.write_avro(path) Nrr)rrr r1r write_avro)rr compressionrs rrDataFrame.write_avroE sI@  (K dS$K ( (%d+D <D Dt4rA1r)position table_style table_namecolumn_formats dtype_formatsconditional_formats header_format column_totals column_widths row_totals row_heights sparklinesformulasrr autofilterautofithidden_columnshide_gridlines sheet_zoom freeze_panesc|SSKJnJnJnJnJnJnJnJn [SSS9n SSK J n! U"X5un"n#n$XR5n&n%U%n'U"U"5n(U=(d 0nU"U5unn)U=(d U"U"5nU"U%U(UU UU UUU UUS9 un*nn%[U[5(aU!"U5OUn+U+SU%R-[!U&5-[!U(+5- [![#U 55-U+SU%R$-S- 4n,S n-S n.U,SU-:d U,SU.:a2S U%RS U%R$S U<SU-SU.S3 n/['U/5eU&(aU(aWU#R("/U+QU,QU%R+5UU*UU[#U 5=(a U&(+US.U)EP76 U(a U"U%U#UU+UU(S9 Uc [-5n0O.[U[5(aU1n0O[-[/U'U55n0U(aBU&(d;U R0n1[3U15S:aSU13n/[5U/5eU#R75 [U [ 5(a![8R;U%R<U 5n O [?U'U SSS9n U"U =(d 05n U%R<Hdn2U2U0;aSS0O0n3U+SU%RAU25-n4U2U ;aU#RCU4U4U U2SU35 MFU3(dMOU#REU4U4SSU35 Mf U=(d 0RG5Hun2n5U"U#U%U+U2UU5S9 M U(aU#RIS5 U(aU#RKU5 U (a[U [ 5(a/[MU+SU,SS-5Hn6U#ROU6U 5 M OG[U [85(a2U"U 5RG5Hun6n7U#ROU6U75 M U(a6[U[5(aU#RQU5 OU#RP"U6 U$(aU"RS5 U"$)a? Write frame data to a table in an Excel workbook/worksheet. Parameters ---------- workbook : {str, Workbook} String name or path of the workbook to create, BytesIO object, file opened in binary-mode, or an `xlsxwriter.Workbook` object that has not been closed. If None, writes to a `dataframe.xlsx` workbook in the working directory. worksheet : {str, Worksheet} Name of target worksheet or an `xlsxwriter.Worksheet` object (in which case `workbook` must be the parent `xlsxwriter.Workbook` object); if None, writes to "Sheet1" when creating a new workbook (note that writing to an existing workbook requires a valid existing -or new- worksheet name). position : {str, tuple} Table position in Excel notation (eg: "A1"), or a (row,col) integer tuple. table_style : {str, dict} A named Excel table style, such as "Table Style Medium 4", or a dictionary of `{"key":value,}` options containing one or more of the following keys: "style", "first_column", "last_column", "banded_columns, "banded_rows". table_name : str Name of the output table object in the worksheet; can then be referred to in the sheet by formulae/charts, or by subsequent `xlsxwriter` operations. column_formats : dict A `{colname(s):str,}` or `{selector:str,}` dictionary for applying an Excel format string to the given columns. Formats defined here (such as "dd/mm/yyyy", "0.00%", etc) will override any defined in `dtype_formats`. dtype_formats : dict A `{dtype:str,}` dictionary that sets the default Excel format for the given dtype. (This can be overridden on a per-column basis by the `column_formats` param). conditional_formats : dict A dictionary of colname (or selector) keys to a format str, dict, or list that defines conditional formatting options for the specified columns. * If supplying a string typename, should be one of the valid `xlsxwriter` types such as "3_color_scale", "data_bar", etc. * If supplying a dictionary you can make use of any/all `xlsxwriter` supported options, including icon sets, formulae, etc. * Supplying multiple columns as a tuple/key will apply a single format across all columns - this is effective in creating a heatmap, as the min/max values will be determined across the entire range, not per-column. * Finally, you can also supply a list made up from the above options in order to apply *more* than one conditional format to the same range. header_format : dict A `{key:value,}` dictionary of `xlsxwriter` format options to apply to the table header row, such as `{"bold":True, "font_color":"#702963"}`. column_totals : {bool, list, dict} Add a column-total row to the exported table. * If True, all numeric columns will have an associated total using "sum". * If passing a string, it must be one of the valid total function names and all numeric columns will have an associated total using that function. * If passing a list of colnames, only those given will have a total. * For more control, pass a `{colname:funcname,}` dict. Valid column-total function names are "average", "count_nums", "count", "max", "min", "std_dev", "sum", and "var". column_widths : {dict, int} A `{colname:int,}` or `{selector:int,}` dict or a single integer that sets (or overrides if autofitting) table column widths, in integer pixel units. If given as an integer the same value is used for all table columns. row_totals : {dict, list, bool} Add a row-total column to the right-hand side of the exported table. * If True, a column called "total" will be added at the end of the table that applies a "sum" function row-wise across all numeric columns. * If passing a list/sequence of column names, only the matching columns will participate in the sum. * Can also pass a `{colname:columns,}` dictionary to create one or more total columns with distinct names, referencing different columns. row_heights : {dict, int} An int or `{row_index:int,}` dictionary that sets the height of the given rows (if providing a dictionary) or all rows (if providing an integer) that intersect with the table body (including any header and total row) in integer pixel units. Note that `row_index` starts at zero and will be the header row (unless `include_header` is False). sparklines : dict A `{colname:list,}` or `{colname:dict,}` dictionary defining one or more sparklines to be written into a new column in the table. * If passing a list of colnames (used as the source of the sparkline data) the default sparkline settings are used (eg: line chart with no markers). * For more control an `xlsxwriter`-compliant options dict can be supplied, in which case three additional polars-specific keys are available: "columns", "insert_before", and "insert_after". These allow you to define the source columns and position the sparkline(s) with respect to other table columns. If no position directive is given, sparklines are added to the end of the table (eg: to the far right) in the order they are given. formulas : dict A `{colname:formula,}` or `{colname:dict,}` dictionary defining one or more formulas to be written into a new column in the table. Note that you are strongly advised to use structured references in your formulae wherever possible to make it simple to reference columns by name. * If providing a string formula (such as "=[@colx]*[@coly]") the column will be added to the end of the table (eg: to the far right), after any default sparklines and before any row_totals. * For the most control supply an options dictionary with the following keys: "formula" (mandatory), one of "insert_before" or "insert_after", and optionally "return_dtype". The latter is used to appropriately format the output of the formula and allow it to participate in row/column totals. float_precision : int Default number of decimals displayed for floating point columns (note that this is purely a formatting directive; the actual values are not rounded). include_header : bool Indicate if the table should be created with a header row. autofilter : bool If the table has headers, provide autofilter capability. autofit : bool Calculate individual column widths from the data. hidden_columns : str | list A column name, list of column names, or a selector representing table columns to mark as hidden in the output worksheet. hide_gridlines : bool Do not display any gridlines on the output worksheet. sheet_zoom : int Set the default zoom level of the output worksheet. freeze_panes : str | (str, int, int) | (int, int) | (int, int, int, int) Freeze workbook panes. * If (row, col) is supplied, panes are split at the top-left corner of the specified cell, which are 0-indexed. Thus, to freeze only the top row, supply (1, 0). * Alternatively, cell notation can be used to supply the cell. For example, "A2" indicates the split occurs at the top-left of cell A2, which is the equivalent of (1, 0). * If (row, col, top_row, top_col) are supplied, the panes are split based on the `row` and `col`, and the scrolling region is initialized to begin at the `top_row` and `top_col`. Thus, to freeze only the top row and have the scrolling region begin at row 10, column D (5th col), supply (1, 0, 9, 4). Using cell notation for (row, col), supplying ("A2", 9, 4) is equivalent. Notes ----- * A list of compatible `xlsxwriter` format property names can be found here: https://xlsxwriter.readthedocs.io/format.html#format-methods-and-format-properties * Conditional formatting dictionaries should provide xlsxwriter-compatible definitions; polars will take care of how they are applied on the worksheet with respect to the relative sheet/column position. For supported options, see: https://xlsxwriter.readthedocs.io/working_with_conditional_formats.html * Similarly, sparkline option dictionaries should contain xlsxwriter-compatible key/values, as well as a mandatory polars "columns" key that defines the sparkline source data; these source columns should all be adjacent. Two other polars-specific keys are available to help define where the sparkline appears in the table: "insert_after", and "insert_before". The value associated with these keys should be the name of a column in the exported table. https://xlsxwriter.readthedocs.io/working_with_sparklines.html * Formula dictionaries *must* contain a key called "formula", and then optional "insert_after", "insert_before", and/or "return_dtype" keys. These additional keys allow the column to be injected into the table at a specific location, and/or to define the return type of the formula (eg: "Int64", "Float64", etc). Formulas that refer to table columns should use Excel's structured references syntax to ensure the formula is applied correctly and is table-relative. https://support.microsoft.com/en-us/office/using-structured-references-with-excel-tables-f5ed2452-2337-4f71-bed3-c8ae6d2b276e Examples -------- Instantiate a basic DataFrame: >>> from random import uniform >>> from datetime import date >>> >>> df = pl.DataFrame( ... { ... "dtm": [date(2023, 1, 1), date(2023, 1, 2), date(2023, 1, 3)], ... "num": [uniform(-500, 500), uniform(-500, 500), uniform(-500, 500)], ... "val": [10_000, 20_000, 30_000], ... } ... ) Export to "dataframe.xlsx" (the default workbook name, if not specified) in the working directory, add column totals ("sum" by default) on all numeric columns, then autofit: >>> df.write_excel(column_totals=True, autofit=True) # doctest: +SKIP Write frame to a specific location on the sheet, set a named table style, apply US-style date formatting, increase default float precision, apply a non-default total function to a single column, autofit: >>> df.write_excel( # doctest: +SKIP ... position="B4", ... table_style="Table Style Light 16", ... dtype_formats={pl.Date: "mm/dd/yyyy"}, ... column_totals={"num": "average"}, ... float_precision=6, ... autofit=True, ... ) Write the same frame to a named worksheet twice, applying different styles and conditional formatting to each table, adding table titles using explicit xlsxwriter integration: >>> from xlsxwriter import Workbook >>> with Workbook("multi_frame.xlsx") as wb: # doctest: +SKIP ... # basic/default conditional formatting ... df.write_excel( ... workbook=wb, ... worksheet="data", ... position=(3, 1), # specify position as (row,col) coordinates ... conditional_formats={"num": "3_color_scale", "val": "data_bar"}, ... table_style="Table Style Medium 4", ... ) ... ... # advanced conditional formatting, custom styles ... df.write_excel( ... workbook=wb, ... worksheet="data", ... position=(df.height + 7, 1), ... table_style={ ... "style": "Table Style Light 4", ... "first_column": True, ... }, ... conditional_formats={ ... "num": { ... "type": "3_color_scale", ... "min_color": "#76933c", ... "mid_color": "#c4d79b", ... "max_color": "#ebf1de", ... }, ... "val": { ... "type": "data_bar", ... "data_bar_2010": True, ... "bar_color": "#9bbb59", ... "bar_negative_color_same": True, ... "bar_negative_border_color_same": True, ... }, ... }, ... column_formats={"num": "#,##0.000;[White]-#,##0.000"}, ... column_widths={"val": 125}, ... autofit=True, ... ) ... ... # add some table titles (with a custom format) ... ws = wb.get_worksheet_by_name("data") ... fmt_title = wb.add_format( ... { ... "font_color": "#4f6228", ... "font_size": 12, ... "italic": True, ... "bold": True, ... } ... ) ... ws.write(2, 1, "Basic/default conditional formatting", fmt_title) ... ws.write( ... df.height + 6, 1, "Customised conditional formatting", fmt_title ... ) Export a table containing two different types of sparklines. Use default options for the "trend" sparkline and customized options (and positioning) for the "+/-" win_loss sparkline, with non-default integer dtype formatting, column totals, a subtle two-tone heatmap and hidden worksheet gridlines: >>> df = pl.DataFrame( ... { ... "id": ["aaa", "bbb", "ccc", "ddd", "eee"], ... "q1": [100, 55, -20, 0, 35], ... "q2": [30, -10, 15, 60, 20], ... "q3": [-50, 0, 40, 80, 80], ... "q4": [75, 55, 25, -10, -55], ... } ... ) >>> df.write_excel( # doctest: +SKIP ... table_style="Table Style Light 2", ... # apply accounting format to all flavours of integer ... dtype_formats={dt: "#,##0_);(#,##0)" for dt in [pl.Int32, pl.Int64]}, ... sparklines={ ... # default options; just provide source cols ... "trend": ["q1", "q2", "q3", "q4"], ... # customized sparkline type, with positioning directive ... "+/-": { ... "columns": ["q1", "q2", "q3", "q4"], ... "insert_after": "id", ... "type": "win_loss", ... }, ... }, ... conditional_formats={ ... # create a unified multi-column heatmap ... ("q1", "q2", "q3", "q4"): { ... "type": "2_color_scale", ... "min_color": "#95b3d7", ... "max_color": "#ffffff", ... }, ... }, ... column_totals=["q1", "q2", "q3", "q4"], ... row_totals=True, ... hide_gridlines=True, ... ) Export a table containing an Excel formula-based column that calculates a standardised Z-score, showing use of structured references in conjunction with positioning directives, column totals, and custom formatting. >>> df = pl.DataFrame( ... { ... "id": ["a123", "b345", "c567", "d789", "e101"], ... "points": [99, 45, 50, 85, 35], ... } ... ) >>> df.write_excel( # doctest: +SKIP ... table_style={ ... "style": "Table Style Medium 15", ... "first_column": True, ... }, ... column_formats={ ... "id": {"font": "Consolas"}, ... "points": {"align": "center"}, ... "z-score": {"align": "center"}, ... }, ... column_totals="average", ... formulas={ ... "z-score": { ... # use structured references to refer to the table columns and 'totals' row ... "formula": "=STANDARDIZE([@points], [[#Totals],[points]], STDEV([points]))", ... "insert_after": "points", ... "return_dtype": pl.Float64, ... } ... }, ... hide_gridlines=True, ... sheet_zoom=125, ... ) Create and reference a Worksheet object directly, adding a basic chart. Taking advantage of structured references to set chart series values and categories is strongly recommended so that you do not have to calculate cell positions with respect to the frame data and worksheet: >>> with Workbook("basic_chart.xlsx") as wb: # doctest: +SKIP ... # create worksheet object and write frame data to it ... ws = wb.add_worksheet("demo") ... df.write_excel( ... workbook=wb, ... worksheet=ws, ... table_name="DataTable", ... table_style="Table Style Medium 26", ... hide_gridlines=True, ... ) ... # create chart object, point to the written table ... # data using structured references, and style it ... chart = wb.add_chart({"type": "column"}) ... chart.set_title({"name": "Example Chart"}) ... chart.set_legend({"none": True}) ... chart.set_style(38) ... chart.add_series( ... { # note the use of structured references ... "values": "=DataTable[points]", ... "categories": "=DataTable[id]", ... "data_labels": {"value": True}, ... } ... ) ... # add chart to the worksheet ... ws.insert_chart("D1", chart) r)_unpack_multi_column_dict_xl_apply_conditional_formats_xl_inject_sparklines_xl_setup_table_columns_xl_setup_table_options_xl_setup_workbook_xl_unique_table_name_XLFormatCache xlsxwriterzExcel export requires) err_prefix)xl_cell_to_rowcol) r format_cacherrrrrrrrrrii@zwriting xz frame at z& does not fit worksheet dimensions of z rows and z columns)rrr header_rowr total_rowr)rwsr table_startrrN)rrr|z:`autofit=True` requires xlsxwriter 3.0.8 or higher, found TF) expand_keys expand_valueshidden)rparamsr)*"polars.io.spreadsheet._write_utilsrrrrrrrrrWxlsxwriter.utilityrr5rrrrr[rr] add_tabler-setrfrr2r^rrfromkeysrreget_column_indexset_column_pixels set_columnrar set_zoomrYset_row_pixelsr close)8rworkbook worksheetrrrrrrrrrrrrrrrrrr r r r rrrrrrrrrrwbr can_closerr5 df_original fmt_cache table_options table_columnsr table_finishexcel_max_valid_rowsexcel_max_valid_colsrr!xlvroptionscol_idxr"r>rs8 r write_excelDataFrame.write_exceln sL %\>UV 8/xCB ]]_H  #2& '-2%<[%I" ]<#8#< ,C")'''+#!! - ) ~r",6h+D+D h '(  Nii (m n$% &$}%&  ' NRXX % )   '$ O2 2A!55RYYKq *XLHnpDoEEOPdOeemnC', ,> LL   GGI(,"0",!%m!4!EX& $   #-(; +#1!*   !UF  , ,$%F*;GHF 8((CS!I-RSVRWX055 JJL mS ) ) MM"**mDM2]EM2-2E2F jjF*0F*:x&G!!nr':':6'BBG&$$!&)  gwdGD! */R668NFF !-  9    a  KK # +s++ Qa11DEC%%c;7FK..#<[#I#O#O#QKC%%c62$R ,,, -.  HHJ r)rrrrrcgrrrrrrrrrs r write_ipcDataFrame.write_ipcsrcgrrr?s rr@rAsrc USLnUc [5nOUnSSKJn UR5R UUUUUUU R 5SS9 U(aU$S$)a Write to Arrow IPC binary stream or Feather file. See "File or Random Access format" in https://arrow.apache.org/docs/python/ipc.html. .. versionchanged:: 1.1 The `future` parameter was renamed `compat_level`. Parameters ---------- file Path or writable file-like object to which the IPC data will be written. If set to `None`, the output is returned as a BytesIO object. compression : {'uncompressed', 'lz4', 'zstd'} Compression method. Defaults to "uncompressed". compat_level Use a specific compatibility level when exporting Polars' internal data structures. storage_options Options that indicate how to connect to a cloud provider. The cloud providers currently supported are AWS, GCP, and Azure. See supported keys here: * `aws `_ * `gcp `_ * `azure `_ * Hugging Face (`hf://`): Accepts an API key under the `token` parameter: `{'token': '...'}`, or by setting the `HF_TOKEN` environment variable. If `storage_options` is not provided, Polars will try to infer the information from environment variables. credential_provider Provide a function that can be called to provide cloud storage credentials. The function is expected to return a dictionary of credential keys along with an optional credential expiry time. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. retries Number of retries if accessing a cloud instance fails. Examples -------- >>> import pathlib >>> >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3, 4, 5], ... "bar": [6, 7, 8, 9, 10], ... "ham": ["a", "b", "c", "d", "e"], ... } ... ) >>> path: pathlib.Path = dirpath / "new_file.arrow" >>> df.write_ipc(path) Nrrr)rrrrrrr)r rrrsink_ipcr) rrrrrrr return_bytesrRrs rr@rAskLt| <YFF<  #%+ 3'..0  &v/4/r)rrcgrrrrrrs rwrite_ipc_streamDataFrame.write_ipc_streamsrcgrrrGs rrHrIrc USLnU(a [5nO&[U[[45(a [ U5nUcSnO![U[ 5(a UR nUcSnURRXW5 U(aU$S$)aj Write to Arrow IPC record batch stream. See "Streaming format" in https://arrow.apache.org/docs/python/ipc.html. .. versionchanged:: 1.1 The `future` parameter was renamed `compat_level`. Parameters ---------- file Path or writable file-like object to which the IPC record batch data will be written. If set to `None`, the output is returned as a BytesIO object. compression : {'uncompressed', 'lz4', 'zstd'} Compression method. Defaults to "uncompressed". compat_level Use a specific compatibility level when exporting Polars' internal data structures. Examples -------- >>> import pathlib >>> >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3, 4, 5], ... "bar": [6, 7, 8, 9, 10], ... "ham": ["a", "b", "c", "d", "e"], ... } ... ) >>> path: pathlib.Path = dirpath / "new_file.arrow" >>> df.write_ipc_stream(path) NTr) r rrr r1rcrrrH)rrrrrErs rrHrI'sRt| 9D sDk * *%d+D  "O  k 2 2*33O  (K !!$_E#t--rzstdl)rcompression_level statisticsrow_group_sizedata_page_sizer3pyarrow_options partition_bypartition_chunk_size_bytesrrrmetadatamkdirc(UcSn[U[[45(a2U cUb!URS5(a [ USS9nO [ U5nU(GaBUS:Xd[U[ 5(a Sn[ U5eUb Sn[ U5eU(a S n[ U5eUR5n0n[U5H)unnURcS U3O URnUUU'M+ [R"U5nS SK nUc0nUS:XaSOUUS 'X8S 'XHS'XXS'XhS'URS5(a-[RR"SUUS.U=(d 0D6 g[RR"SUUS.U=(d 0D6 gUnSnU b2[U[5(d Sn[!U5eS SKJn U"XS9nSnSnS SKJn UR+5R-UUUUUUU U U UUUUR/5S9 g)aP Write to Apache Parquet file. Parameters ---------- file File path or writable file-like object to which the result will be written. This should be a path to a directory if writing a partitioned dataset. compression : {'lz4', 'uncompressed', 'snappy', 'gzip', 'lzo', 'brotli', 'zstd'} Choose "zstd" for good compression performance. Choose "lz4" for fast compression/decompression. Choose "snappy" for more backwards compatibility guarantees when you deal with older parquet readers. compression_level The level of compression to use. Higher compression means smaller files on disk. - "gzip" : min-level: 0, max-level: 9. - "brotli" : min-level: 0, max-level: 11. - "zstd" : min-level: 1, max-level: 22. statistics Write statistics to the parquet headers. This is the default behavior. Possible values: - `True`: enable default set of statistics (default). Some statistics may be disabled. - `False`: disable all statistics - "full": calculate and write all available statistics. Cannot be combined with `use_pyarrow`. - `{ "statistic-key": True / False, ... }`. Cannot be combined with `use_pyarrow`. Available keys: - "min": column minimum value (default: `True`) - "max": column maximum value (default: `True`) - "distinct_count": number of unique column values (default: `False`) - "null_count": number of null values in column (default: `True`) row_group_size Size of the row groups in number of rows. Defaults to 512^2 rows. data_page_size Size of the data page in bytes. Defaults to 1024^2 bytes. use_pyarrow Use C++ parquet implementation vs Rust parquet implementation. At the moment C++ supports more features. pyarrow_options Arguments passed to `pyarrow.parquet.write_table`. If you pass `partition_cols` here, the dataset will be written using `pyarrow.parquet.write_to_dataset`. The `partition_cols` parameter leads to write the dataset to a directory. Similar to Spark's partitioned datasets. partition_by Column(s) to partition by. A partitioned dataset will be written if this is specified. This parameter is considered unstable and is subject to change. partition_chunk_size_bytes Approximate size to split DataFrames within a single partition when writing. Note this is calculated using the size of the DataFrame in memory - the size of the output file may differ depending on the file format / compression. storage_options Options that indicate how to connect to a cloud provider. The cloud providers currently supported are AWS, GCP, and Azure. See supported keys here: * `aws `_ * `gcp `_ * `azure `_ * Hugging Face (`hf://`): Accepts an API key under the `token` parameter: `{'token': '...'}`, or by setting the `HF_TOKEN` environment variable. If `storage_options` is not provided, Polars will try to infer the information from environment variables. credential_provider Provide a function that can be called to provide cloud storage credentials. The function is expected to return a dictionary of credential keys along with an optional credential expiry time. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. retries Number of retries if accessing a cloud instance fails. metadata A dictionary or callback to add key-values to the file-level Parquet metadata. .. warning:: This functionality is considered **experimental**. It may be removed or changed at any point without it being considered a breaking change. mkdir: bool Recursively create all the directories in the path. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. Examples -------- >>> import pathlib >>> >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3, 4, 5], ... "bar": [6, 7, 8, 9, 10], ... "ham": ["a", "b", "c", "d", "e"], ... } ... ) >>> path: pathlib.Path = dirpath / "new_file.parquet" >>> df.write_parquet(path) We can use pyarrow with use_pyarrow_write_to_dataset=True to write partitioned datasets. The following example will write the first row to ../watermark=1/*.parquet and the other rows to ../watermark=2/*.parquet. >>> df = pl.DataFrame({"a": [1, 2, 3], "watermark": [1, 2, 2]}) >>> path: pathlib.Path = dirpath / "partitioned_object" >>> df.write_parquet( ... path, ... use_pyarrow=True, ... pyarrow_options={"partition_cols": ["watermark"]}, ... ) Nrpartition_colsF)check_not_directoryfullzQwrite_parquet with `use_pyarrow=True` allows only boolean values for `statistics`zHwrite_parquet with `use_pyarrow=True` cannot be combined with `metadata`zEwrite_parquet with `use_pyarrow=True` cannot be combined with `mkdir`column_rrrNwrite_statisticsrPrQ)r root_path)rwhererz5expected file to be a `str` since partition-by is set)PartitionByKey)byT streamingr) rrNrOrPrQrrrrUrrVrr)rrr rr1rrrr`_namerrpyarrow.parquetparquetwrite_to_dataset write_tabler polars.ior_rrr sink_parquetr)rrrrNrOrPrQr3rRrSrTrrrrUrVrrrrerrr[rRrr_rs r write_parquetDataFrame.write_parquetbsGd  (K dS$K ( ('+0C0CDT0U0U)$EJ)$/ V#z*d'C'Ci o%#` o%] o%--/CD&s^ 6(. (<}&,,#T , ((4.C #&"$#~5; M *4E/ 02<. /0>, -0>, -""#344 ++"',"   &&'," >B(  #dC((Mn$ 0#D:FE F<   #/!))+ 3'..0 ! rfail)if_table_existsrengine_optionsc  U[[5=n;a,SRSU55nSUSU<3n[U5e[ U5R R SS5Sn Uc6[U[5(dU S :XaS nOU RS 5(aS nS9S jn US :XGa;SS K J n J n J n [U[5(a U "U5S 4OUS4up[S5n[UURR 5(dSU<3n[#U5e[%USS5n['U5nUS:XaSnO:US:XaUS:aSU3n[)U5eSnOUS:XaSnOSU<S3n[U5eU(aUO[*R,"5 UR/5nU "U5unnn[U[5(aU "U5OU nU "USS9n[%USS5n['U5nUR S5SS:Xa,USnnUS:a"US:aUS:XaUR1S U35 SnUS!:aUOUR35nUS:a(US":a"UR4"U4UUUUS#.U=(d 0D6nOJUb(UR7SS$5nS%US&US'U3n[)U5eUR4"S:UUUS(.U=(d 0D6nUR95 SSS5 SSS5 W$US :XGa6[:(d S)n[=U5e['[>R@5=nS*:aS+[>R@<3n[)U5e[S US,:aS-OS.S/S09 SS1K!J"n J#n! SS2K$J%n" [U[5(a U!"U5n#OG[UU"5(aURM5n#O%[UU 5(aUn#OSU<3n[#U5eU "U5unnnU(aS3U<S43n[U5eUROS S59RP"S:UUU#USS6.U=(d 0D6n$U$cS$U$$[U[5(aS7U<S83n[U5eSU<3n[#U5e!,(df  GN=f!,(df  W$=f);a? Write the data in a Polars DataFrame to a database. .. versionadded:: 0.20.26 Support for instantiated connection objects in addition to URI strings, and a new `engine_options` parameter. Parameters ---------- table_name Schema-qualified name of the table to create or append to in the target SQL database. If your table name contains special characters, it should be quoted. connection An existing SQLAlchemy or ADBC connection against the target database, or a URI string that will be used to instantiate such a connection, such as: * "postgresql://user:pass@server:port/database" * "sqlite:////path/to/database.db" if_table_exists : {'append', 'replace', 'fail'} The insert mode: * 'replace' will create a new database table, overwriting an existing one. * 'append' will append to an existing table. * 'fail' will fail if table already exists. engine : {'sqlalchemy', 'adbc'} Select the engine to use for writing frame data; only necessary when supplying a URI string (defaults to 'sqlalchemy' if unset) engine_options Additional options to pass to the insert method associated with the engine specified by the option `engine`. * Setting `engine` to "sqlalchemy" currently inserts using Pandas' `to_sql` method (though this will eventually be phased out in favor of a native solution). * Setting `engine` to "adbc" inserts using the ADBC cursor's `adbc_ingest` method. Examples -------- Insert into a temporary table using a PostgreSQL URI and the ADBC engine: >>> df.write_database( ... table_name="target_table", ... connection="postgresql://user:pass@server:port/database", ... engine="adbc", ... engine_options={"temporary": True}, ... ) # doctest: +SKIP Insert into a table using a `pyodbc` SQLAlchemy connection to SQL Server that was instantiated with "fast_executemany=True" to improve performance: >>> pyodbc_uri = ( ... "mssql+pyodbc://user:pass@server:1433/test?" ... "driver=ODBC+Driver+18+for+SQL+Server" ... ) >>> engine = create_engine(pyodbc_uri, fast_executemany=True) # doctest: +SKIP >>> df.write_database( ... table_name="target_table", ... connection=engine, ... ) # doctest: +SKIP Returns ------- int The number of rows affected, if the driver provides this information. Otherwise, returns -1. rVc38# UHn[U5v M g7fr)repr).0ms r +DataFrame.write_database..sC1BAQ1Bsz1write_database `if_table_exists` must be one of {z}, got r}rrN sqlalchemyadbccSSKJn [U"U/SS95n[U5S:aSUS3n[ U5eS/S[U5- -U-upEnXEU4$) zEUnpack optionally qualified table name to catalog/schema/table tuple.r)readerr}) delimiterrz%`table_name` appears to be invalid: ''N)csvrxnextrr)rdelimited_read componentsrcatalogrrs runpack_table_name3DataFrame.write_database..unpack_table_namesg 4+/vQT0U+VJ:"=dV1E o%%)Fa#j/.A$Bj#P GSC' 'r)_get_adbc_module_name_from_uri_import_optional_adbc_driver_open_adbc_connectionTFadbc_driver_managerzunrecognised connection type rz0.0rkcreater)rzB`if_table_exists = 'replace'` requires ADBC version >= 0.7, found rz(unexpected value for `if_table_exists`: z- Choose one of {'fail', 'replace', 'append'})dbapi_submodule_sqlite)r zDROP TABLE IF EXISTS )r)rr|)rmode catalog_namedb_schema_name-zYuse of schema-qualified table names requires adbc-driver-manager version >= 0.7.0, found z and z version >= 0.8.0, found )rrrz]writing with 'sqlalchemy' engine currently requires pandas. Install with: pip install pandasr{z?writing with 'sqlalchemy' engine requires pandas >= 1.5; found )rrz2.0z1.4zpandas >= 2.2 requires) module_name min_versionmin_err_prefix) Connectable create_engine)Sessionz@Unexpected three-part table name; provide the database/catalog (z) on the connection URIrx)rrcon if_existsrzengine z is not supported)rrrz"tuple[str | None, str | None, str]r))rrrrr __module__splitrr startswithpolars.io.database._utilsrrrrWdbapi Connectionrgetattrr2r^r]r^cursorexecuter adbc_ingestrcommitrOrrrsqlalchemy.enginerrsqlalchemy.ormr connectionrto_sql)%rrrrlrrmvalid_write_modesallowedrconnection_module_rootrrrrconncan_close_conndriver_managerdriver_manager_str_versiondriver_manager_versionrrr db_schemaunpacked_table_nameadbc_module_name adbc_driveradbc_driver_str_versionadbc_driver_versionrn_rowsadbc_driver_pypi_name pd_versionrrr sa_objectress% rwrite_databaseDataFrame.write_databasePsHZ 8M#M#4 NiiC1BCCGFwixXgWjkCS/ !!%j!1!*c**.D .T%'226:: ( V   j#..'z2D9 %( !D --BCNdN$8$8$C$CDD5j^Dn$)0PU)V &%23M%N "&(  I-)F2!!; <>5S99  H,?>QGI!o%'J,B,B,DD :KJ:W7$7 "*c223:>/! ;$e +2+}e*T'&34K&L##))#.r2h>)2DYG /&8/'9+y8)>zl'KL' 6?tT]]_ *V38Kv8U#//+!!%,'0  */R F*,<,D,DS#,N)G56e}? 8  LL  OOD !r)roverwrite_schemarrdelta_write_optionscgrr)rrRrrrrrs r write_deltaDataFrame.write_deltasr)rrrcgrr)rrRrrrrdelta_merge_optionss rrrs'*rerror)rrrrrrcUb [SSS9 SSKJnJn Jn U "5 SSKJn Jn U"UR5 [U[[45(aU "[U5SS 9nSS K J n SS KJn [X5(d U "XQUS 5nOUbUS :wa Sn[!U5eSnA0nU(a(UR#5=n(aU"U5=(d 0nUcUb0U=(d 0EUEOSnUS:XaAUc Sn[!U5e[U[5(aU "XS9nOUnUR$"U40UD6$Uc0nU(aSUS'U "SUUUUS.UD6 g)aP Write DataFrame as delta table. Parameters ---------- target URI of a table or a DeltaTable object. mode : {'error', 'append', 'overwrite', 'ignore', 'merge'} How to handle existing data. - If 'error', throw an error if the table already exists (default). - If 'append', will add new data. - If 'overwrite', will replace table with new data. - If 'ignore', will not write anything if table already exists. - If 'merge', return a `TableMerger` object to merge data from the DataFrame with the existing data. overwrite_schema If True, allows updating the schema of the table. .. deprecated:: 0.20.14 Use the parameter `delta_write_options` instead and pass `{"schema_mode": "overwrite"}`. storage_options Extra options for the storage backends supported by `deltalake`. For cloud storages, this may include configurations for authentication etc. - See a list of supported storage options for S3 `here `__. - See a list of supported storage options for GCS `here `__. - See a list of supported storage options for Azure `here `__. credential_provider Provide a function that can be called to provide cloud storage credentials. The function is expected to return a dictionary of credential keys along with an optional credential expiry time. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. delta_write_options Additional keyword arguments while writing a Delta lake Table. See a list of supported write options `here `__. delta_merge_options Keyword arguments which are required to `MERGE` a Delta lake Table. See a list of supported merge options `here `__. Raises ------ TypeError If the DataFrame contains unsupported data types. ArrowInvalidError If the DataFrame contains data types that could not be cast to their primitive type. TableNotFoundError If the delta table doesn't exist and MERGE action is triggered Notes ----- The Polars data types :class:`Null` and :class:`Time` are not supported by the delta protocol specification and will raise a TypeError. Columns using The :class:`Categorical` data type will be converted to normal (non-categorical) strings when written. Polars columns are always nullable. To write data to a delta table with non-nullable columns, a custom pyarrow schema has to be passed to the `delta_write_options`. See the last example below. Examples -------- Write a dataframe to the local filesystem as a Delta Lake table. >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3, 4, 5], ... "bar": [6, 7, 8, 9, 10], ... "ham": ["a", "b", "c", "d", "e"], ... } ... ) >>> table_path = "/path/to/delta-table/" >>> df.write_delta(table_path) # doctest: +SKIP Append data to an existing Delta Lake table on the local filesystem. Note that this will fail if the schema of the new data does not match the schema of the existing table. >>> df.write_delta(table_path, mode="append") # doctest: +SKIP Overwrite a Delta Lake table as a new version. If the schemas of the new and old data are the same, specifying the `schema_mode` is not required. >>> existing_table_path = "/path/to/delta-table/" >>> df.write_delta( ... existing_table_path, ... mode="overwrite", ... delta_write_options={"schema_mode": "overwrite"}, ... ) # doctest: +SKIP Write a DataFrame as a Delta Lake table to a cloud object store like S3. >>> table_path = "s3://bucket/prefix/to/delta-table/" >>> df.write_delta( ... table_path, ... storage_options={ ... "AWS_REGION": "THE_AWS_REGION", ... "AWS_ACCESS_KEY_ID": "THE_AWS_ACCESS_KEY_ID", ... "AWS_SECRET_ACCESS_KEY": "THE_AWS_SECRET_ACCESS_KEY", ... }, ... ) # doctest: +SKIP Write DataFrame as a Delta Lake table with non-nullable columns. >>> import pyarrow as pa >>> existing_table_path = "/path/to/delta-table/" >>> df.write_delta( ... existing_table_path, ... delta_write_options={ ... "schema": pa.schema([pa.field("foo", pa.int64(), nullable=False)]) ... }, ... ) # doctest: +SKIP Write DataFrame as a Delta Lake table with zstd compression. For all `delta_write_options` keyword arguments, check the deltalake docs `here `__, and for Writer Properties in particular `here `__. >>> import deltalake >>> df.write_delta( ... table_path, ... delta_write_options={ ... "writer_properties": deltalake.WriterProperties(compression="zstd"), ... }, ... ) # doctest: +SKIP Merge the DataFrame with an existing Delta Lake table. For all `TableMerger` methods, check the deltalake docs `here `__. >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3, 4, 5], ... "bar": [6, 7, 8, 9, 10], ... "ham": ["a", "b", "c", "d", "e"], ... } ... ) >>> table_path = "/path/to/delta-table/" >>> ( ... df.write_delta( ... "table_path", ... mode="merge", ... delta_merge_options={ ... "predicate": "s.foo = t.foo", ... "source_alias": "s", ... "target_alias": "t", ... }, ... ) ... .when_matched_update_all() ... .when_not_matched_insert_all() ... .execute() ... ) # doctest: +SKIP Nzthe parameter `overwrite_schema` for `write_delta` is deprecated. Use the parameter `delta_write_options` instead and pass `{"schema_mode": "overwrite"}`.0.20.14rr)_check_for_unsupported_types_check_if_delta_available_resolve_delta_lake_uri) DeltaTablewrite_deltalakeFr)!_init_credential_provider_builder)+_get_credentials_from_provider_expiry_awarerrz?cannot use credential_provider when passing a DeltaTable objectmergezYyou need to pass delta_merge_options with at least a given predicate for `MERGE` to work.) table_urirr schema_mode) table_or_urirrrr)r&polars.io.deltarrr deltalakerrrrrr ,polars.io.cloud.credential_provider._builderr.polars.io.cloud.credential_provider._providersrrbuild_credential_providerr)rrRrrrrrrrrrrrrrcredential_provider_builderrcredential_provider_credsproviderdts rrrs\  ' %l!    "#9$T[[1 fsDk * *,S[GF  &--*K#_m+ '! ,1D1NSCS/ !*. ' $&! &3MMO OH O>> df = pl.DataFrame( ... { ... "x": list(reversed(range(1_000_000))), ... "y": [v / 1000 for v in range(1_000_000)], ... "z": [str(v) for v in range(1_000_000)], ... }, ... schema=[("x", pl.UInt32), ("y", pl.Float64), ("z", pl.String)], ... ) >>> df.estimated_size() 17888890 >>> df.estimated_size("mb") 17.0601749420166 )restimated_sizer5)runitszs rrDataFrame.estimated_sizes"TXX $ $ &2$$r)r header_name column_namescU(aUOSn[U[5(a/[UR5Vs/sHn[ U5PM nnOUnUR UR RXF55$s snf)u Transpose a DataFrame over the diagonal. Parameters ---------- include_header If set, the column names will be added as first column. header_name If `include_header` is set, this determines the name of the column that will be inserted. column_names Optional iterable yielding strings or a string naming an existing column. These will name the value (non-header) columns in the transposed data. Notes ----- This is a very expensive operation. Perhaps you can do it differently. Returns ------- DataFrame Examples -------- >>> df = pl.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]}) >>> df.transpose(include_header=True) shape: (2, 4) ┌────────┬──────────┬──────────┬──────────┐ │ column ┆ column_0 ┆ column_1 ┆ column_2 │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 ┆ i64 │ ╞════════╪══════════╪══════════╪══════════╡ │ a ┆ 1 ┆ 2 ┆ 3 │ │ b ┆ 4 ┆ 5 ┆ 6 │ └────────┴──────────┴──────────┴──────────┘ Replace the auto-generated column names with a list >>> df.transpose(include_header=False, column_names=["x", "y", "z"]) shape: (2, 3) ┌─────┬─────┬─────┐ │ x ┆ y ┆ z │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ 1 ┆ 2 ┆ 3 │ │ 4 ┆ 5 ┆ 6 │ └─────┴─────┴─────┘ Include the header as a separate column >>> df.transpose( ... include_header=True, header_name="foo", column_names=["x", "y", "z"] ... ) shape: (2, 4) ┌─────┬─────┬─────┬─────┐ │ foo ┆ x ┆ y ┆ z │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╪═════╡ │ a ┆ 1 ┆ 2 ┆ 3 │ │ b ┆ 4 ┆ 5 ┆ 6 │ └─────┴─────┴─────┴─────┘ Replace the auto-generated column with column names from a generator function >>> def name_generator(): ... base_name = "my_column_" ... count = 0 ... while True: ... yield f"{base_name}{count}" ... count += 1 >>> df.transpose(include_header=False, column_names=name_generator()) shape: (2, 3) ┌─────────────┬─────────────┬─────────────┐ │ my_column_0 ┆ my_column_1 ┆ my_column_2 │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 │ ╞═════════════╪═════════════╪═════════════╡ │ 1 ┆ 2 ┆ 3 │ │ 4 ┆ 5 ┆ 6 │ └─────────────┴─────────────┴─────────────┘ Use an existing column as the new column names >>> df = pl.DataFrame(dict(id=["i", "j", "k"], a=[1, 2, 3], b=[4, 5, 6])) >>> df.transpose(column_names="id") shape: (2, 3) ┌─────┬─────┬─────┐ │ i ┆ j ┆ k │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ 1 ┆ 2 ┆ 3 │ │ 4 ┆ 5 ┆ 6 │ └─────┴─────┴─────┘ >>> df.transpose(include_header=True, header_name="new_id", column_names="id") shape: (2, 4) ┌────────┬─────┬─────┬─────┐ │ new_id ┆ i ┆ j ┆ k │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 ┆ i64 │ ╞════════╪═════╪═════╪═════╡ │ a ┆ 1 ┆ 2 ┆ 3 │ │ b ┆ 4 ┆ 5 ┆ 6 │ └────────┴─────┴─────┴─────┘ N)rrrYrr|rr transpose)rrrr keep_names_asr column_names_s rrDataFrame.transposesjd(6 4 lI . .9>t{{9KL9KAT,/9KMLM(Mtxx11-OPPMsA<chUR[R"S5R55$)u Reverse the DataFrame. Examples -------- >>> df = pl.DataFrame( ... { ... "key": ["a", "b", "c"], ... "val": [1, 2, 3], ... } ... ) >>> df.reverse() shape: (3, 2) ┌─────┬─────┐ │ key ┆ val │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═════╪═════╡ │ c ┆ 3 │ │ b ┆ 2 │ │ a ┆ 1 │ └─────┴─────┘ r)rJrKrareverser s rrDataFrame.reverseFs$0{{155:--/00rrc~SSKJn UR5RXS9R UR 5S9$)u/ Rename column names. Parameters ---------- mapping Key value pairs that map from old name to new name, or a function that takes the old name as input and returns the new name. strict Validate that all column names exist in the current schema, and throw an exception if any do not. (Note that this parameter is a no-op when passing a function to `mapping`). Examples -------- >>> df = pl.DataFrame( ... {"foo": [1, 2, 3], "bar": [6, 7, 8], "ham": ["a", "b", "c"]} ... ) >>> df.rename({"foo": "apple"}) shape: (3, 3) ┌───────┬─────┬─────┐ │ apple ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═══════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ │ 2 ┆ 7 ┆ b │ │ 3 ┆ 8 ┆ c │ └───────┴─────┴─────┘ >>> df.rename(lambda column_name: "c" + column_name[1:]) shape: (3, 3) ┌─────┬─────┬─────┐ │ coo ┆ car ┆ cam │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ │ 2 ┆ 7 ┆ b │ │ 3 ┆ 8 ┆ c │ └─────┴─────┴─────┘ rrrr)rrrrenamecollectr)rmappingrrs rrDataFrame.rename`s9X = IIK VGV + W=#7#7#9W : rcU=nS:a4URU-nUS:aSUSURS3n[U5eO-XR:aSUSURS3n[U5e[U[R5(a'UR R XR5 U$[U[5(a[R"U5n[U[R5(a?URnURX5 URU5R UlU$SU<S[U5S3n[!U5e)u Insert a Series (or expression) at a certain column index. This operation is in place. Parameters ---------- index Index at which to insert the new column. column `Series` or expression to insert. Examples -------- Insert a new Series column at the given index: >>> df = pl.DataFrame({"foo": [1, 2, 3], "bar": [4, 5, 6]}) >>> s = pl.Series("baz", [97, 98, 99]) >>> df.insert_column(1, s) shape: (3, 3) ┌─────┬─────┬─────┐ │ foo ┆ baz ┆ bar │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ 1 ┆ 97 ┆ 4 │ │ 2 ┆ 98 ┆ 5 │ │ 3 ┆ 99 ┆ 6 │ └─────┴─────┴─────┘ Insert a new expression column at the given index: >>> df = pl.DataFrame( ... {"a": [2, 4, 2], "b": [0.5, 4, 10], "c": ["xx", "yy", "zz"]} ... ) >>> expr = (pl.col("b") / pl.col("a")).alias("b_div_a") >>> df.insert_column(2, expr) shape: (3, 4) ┌─────┬──────┬─────────┬─────┐ │ a ┆ b ┆ b_div_a ┆ c │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ f64 ┆ str │ ╞═════╪══════╪═════════╪═════╡ │ 2 ┆ 0.5 ┆ 0.25 ┆ xx │ │ 4 ┆ 4.0 ┆ 1.0 ┆ yy │ │ 2 ┆ 10.0 ┆ 5.0 ┆ zz │ └─────┴──────┴─────────┴─────┘ rz column index z is out of range (frame has z columns)z%column must be a Series or Expr, got z (type=))r IndexErrorrrrur insert_columnrrrKrarsrrrJr3r)rrroriginal_indexrcolss rrDataFrame.insert_columns<b$ #Nq (JJ&Eqy%n%55QRVR\R\Q]]fg o%ZZ !.!11Mdjj\YbcCS/ ! fbii ( ( HH " "5)) 4 &#&&v&"''**|| E*;;t,00 >fZwObciOjNkklmn$rcSSKJn UR5R"U0UD6R UR 5S9$)u Filter rows, retaining those that match the given predicate expression(s). The original order of the remaining rows is preserved. Only rows where the predicate resolves as True are retained; when the predicate result is False (or null), the row is discarded. Parameters ---------- predicates Expression(s) that evaluate to a boolean Series. constraints Column filters; use `name = value` to filter columns by the supplied value. Each constraint will behave the same as `pl.col(name).eq(value)`, and be implicitly joined with the other filter conditions using `&`. Notes ----- If you are transitioning from Pandas, and performing filter operations based on the comparison of two or more columns, please note that in Polars any comparison involving `null` values will result in a `null` result, *not* boolean True or False. As a result, these rows will not be retained. Ensure that null values are handled appropriately to avoid unexpected behaviour (see examples below). See Also -------- remove Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3, None, 4, None, 0], ... "bar": [6, 7, 8, None, None, 9, 0], ... "ham": ["a", "b", "c", None, "d", "e", "f"], ... } ... ) Filter rows matching a condition: >>> df.filter(pl.col("foo") > 1) shape: (3, 3) ┌─────┬──────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪══════╪═════╡ │ 2 ┆ 7 ┆ b │ │ 3 ┆ 8 ┆ c │ │ 4 ┆ null ┆ d │ └─────┴──────┴─────┘ Filter on multiple conditions, combined with and/or operators: >>> df.filter( ... (pl.col("foo") < 3) & (pl.col("ham") == "a"), ... ) shape: (1, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ └─────┴─────┴─────┘ >>> df.filter( ... (pl.col("foo") == 1) | (pl.col("ham") == "c"), ... ) shape: (2, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ │ 3 ┆ 8 ┆ c │ └─────┴─────┴─────┘ Provide multiple filters using `*args` syntax: >>> df.filter( ... pl.col("foo") <= 2, ... ~pl.col("ham").is_in(["b", "c"]), ... ) shape: (2, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ │ 0 ┆ 0 ┆ f │ └─────┴─────┴─────┘ Provide multiple filters using `**kwargs` syntax: >>> df.filter(foo=2, ham="b") shape: (1, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 2 ┆ 7 ┆ b │ └─────┴─────┴─────┘ Filter by comparing two columns against each other: >>> df.filter( ... pl.col("foo") == pl.col("bar"), ... ) shape: (1, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 0 ┆ 0 ┆ f │ └─────┴─────┴─────┘ >>> df.filter( ... pl.col("foo") != pl.col("bar"), ... ) shape: (3, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ │ 2 ┆ 7 ┆ b │ │ 3 ┆ 8 ┆ c │ └─────┴─────┴─────┘ Notice how the row with `None` values is filtered out. In order to keep the same behavior as pandas, use: >>> df.filter( ... pl.col("foo").ne_missing(pl.col("bar")), ... ) shape: (5, 3) ┌──────┬──────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞══════╪══════╪═════╡ │ 1 ┆ 6 ┆ a │ │ 2 ┆ 7 ┆ b │ │ 3 ┆ 8 ┆ c │ │ 4 ┆ null ┆ d │ │ null ┆ 9 ┆ e │ └──────┴──────┴─────┘ rrr)rrrfilterrrr predicates constraintsrs rrDataFrame.filtersGL = IIK V  0#. 0 W=#7#7#9W : rcSSKJn UR5R"U0UD6R UR 5S9$)u( Remove rows, dropping those that match the given predicate expression(s). The original order of the remaining rows is preserved. Rows where the filter predicate does not evaluate to True are retained (this includes rows where the predicate evaluates as `null`). Parameters ---------- predicates Expression that evaluates to a boolean Series. constraints Column filters; use `name = value` to filter columns using the supplied value. Each constraint behaves the same as `pl.col(name).eq(value)`, and is implicitly joined with the other filter conditions using `&`. Notes ----- If you are transitioning from Pandas, and performing filter operations based on the comparison of two or more columns, please note that in Polars any comparison involving `null` values will result in a `null` result, *not* boolean True or False. As a result, these rows will not be removed. Ensure that null values are handled appropriately to avoid unexpected behaviour (see examples below). See Also -------- filter Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [2, 3, None, 4, 0], ... "bar": [5, 6, None, None, 0], ... "ham": ["a", "b", None, "c", "d"], ... } ... ) Remove rows matching a condition: >>> df.remove(pl.col("bar") >= 5) shape: (3, 3) ┌──────┬──────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞══════╪══════╪══════╡ │ null ┆ null ┆ null │ │ 4 ┆ null ┆ c │ │ 0 ┆ 0 ┆ d │ └──────┴──────┴──────┘ Discard rows based on multiple conditions, combined with and/or operators: >>> df.remove( ... (pl.col("foo") >= 0) & (pl.col("bar") >= 0), ... ) shape: (2, 3) ┌──────┬──────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞══════╪══════╪══════╡ │ null ┆ null ┆ null │ │ 4 ┆ null ┆ c │ └──────┴──────┴──────┘ >>> df.remove( ... (pl.col("foo") >= 0) | (pl.col("bar") >= 0), ... ) shape: (1, 3) ┌──────┬──────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞══════╪══════╪══════╡ │ null ┆ null ┆ null │ └──────┴──────┴──────┘ Provide multiple constraints using `*args` syntax: >>> df.remove( ... pl.col("ham").is_not_null(), ... pl.col("bar") >= 0, ... ) shape: (2, 3) ┌──────┬──────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞══════╪══════╪══════╡ │ null ┆ null ┆ null │ │ 4 ┆ null ┆ c │ └──────┴──────┴──────┘ Provide constraints(s) using `**kwargs` syntax: >>> df.remove(foo=0, bar=0) shape: (4, 3) ┌──────┬──────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞══════╪══════╪══════╡ │ 2 ┆ 5 ┆ a │ │ 3 ┆ 6 ┆ b │ │ null ┆ null ┆ null │ │ 4 ┆ null ┆ c │ └──────┴──────┴──────┘ Remove rows by comparing two columns against each other: >>> df.remove( ... pl.col("foo").ne_missing(pl.col("bar")), ... ) shape: (2, 3) ┌──────┬──────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞══════╪══════╪══════╡ │ null ┆ null ┆ null │ │ 0 ┆ 0 ┆ d │ └──────┴──────┴──────┘ rrr)rrrremoverrrs rrDataFrame.removesGR = IIK V  0#. 0 W=#7#7#9W : r)max_items_per_columnmax_colname_lengthreturn_as_stringcgrrrr r r s rglimpseDataFrame.glimpserKr)r r cgrrrs rrr$srcgrrrs rrr-srr2c 2^^^^[UTR5mTRmS UUUU4SjjnTRR5VVs/sH upVU"XV5PM nnn[ SU55n[ SU55n [ 5n U R STRSTRS35 UH)upn U R SU SU3S U S U 3S U S35 M+ U R5nU(aU$[US S 9 g s snnf)ah Return a dense preview of the DataFrame. The formatting shows one line per column so that wide dataframes display cleanly. Each line shows the column name, the data type, and the first few values. Parameters ---------- max_items_per_column Maximum number of items to show per column. max_colname_length Maximum length of the displayed column names; values that exceed this value are truncated with a trailing ellipsis. return_as_string If True, return the preview as a string instead of printing to stdout. See Also -------- describe, head, tail Examples -------- >>> from datetime import date >>> df = pl.DataFrame( ... { ... "a": [1.0, 2.8, 3.0], ... "b": [4, 5, None], ... "c": [True, False, True], ... "d": [None, "b", "c"], ... "e": ["usd", "eur", None], ... "f": [date(2020, 1, 1), date(2021, 1, 2), date(2022, 1, 1)], ... } ... ) >>> df.glimpse() Rows: 3 Columns: 6 $ a 1.0, 2.8, 3.0 $ b 4, 5, None $ c True, False, True $ d None, 'b', 'c' $ e 'usd', 'eur', None $ f 2020-01-01, 2021-01-02, 2022-01-01 c>^TU[:Xa[O[mTST2U4R5nSR U4SjU55n[ U5T:a USTS- S-nUS[ U5S3U4$)NrVc34># UH nT"U5v M g7frr)rqvfns rrs;DataFrame.glimpse.._parse_column..qs6v!1vsru…<>)rGrprr)rr_dtype_str_repr) col_namer+valuesval_strrr  max_n_valuesrrs @r _parse_column(DataFrame.glimpse.._parse_columnns)V3B-<-12::'9A'=?%Gq!7 8:GC Crc3@# UHun n[U5v M g7frr)rqrrs rrs$DataFrame.glimpse..ysEnh1CMMsc3># UHupn[U5v M g7frr$)rqr dtype_strs rrsr%zsH4aS^^4szRows: z Columns: rz$ r rN)end)rrr+rrztuple[str, str, str]) minrrramaxr rrrprint)rr r r r!rdr+r max_col_name max_col_dtyperrr'rr rs` ` @@rrr6sj/=  D D9= 8I8I8KL8KHA a'8KLEEF H4HI  vdkk]+djj\DE-1 (H LLXa ~-.a !M?9J/K1WIUWX -1 OO  H aT+MsDnearest) interpolationcxUR(d Sn[U5eUR5RXS9$)u6 Summary statistics for a DataFrame. Parameters ---------- percentiles One or more percentiles to include in the summary statistics. All values must be in the range `[0, 1]`. interpolation : {'nearest', 'higher', 'lower', 'midpoint', 'linear', 'equiprobable'} Interpolation method used when calculating percentiles. Notes ----- The median is included by default as the 50% percentile. Warnings -------- We do not guarantee the output of `describe` to be stable. It will show statistics that we deem informative, and may be updated in the future. Using `describe` programmatically (versus interactive exploration) is not recommended for this reason. See Also -------- glimpse Examples -------- >>> from datetime import date, time >>> df = pl.DataFrame( ... { ... "float": [1.0, 2.8, 3.0], ... "int": [40, 50, None], ... "bool": [True, False, True], ... "str": ["zz", "xx", "yy"], ... "date": [date(2020, 1, 1), date(2021, 7, 5), date(2022, 12, 31)], ... "time": [time(10, 20, 30), time(14, 45, 50), time(23, 15, 10)], ... } ... ) Show default frame statistics: >>> df.describe() shape: (9, 7) ┌────────────┬──────────┬──────────┬──────────┬──────┬─────────────────────┬──────────┐ │ statistic ┆ float ┆ int ┆ bool ┆ str ┆ date ┆ time │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ f64 ┆ f64 ┆ f64 ┆ str ┆ str ┆ str │ ╞════════════╪══════════╪══════════╪══════════╪══════╪═════════════════════╪══════════╡ │ count ┆ 3.0 ┆ 2.0 ┆ 3.0 ┆ 3 ┆ 3 ┆ 3 │ │ null_count ┆ 0.0 ┆ 1.0 ┆ 0.0 ┆ 0 ┆ 0 ┆ 0 │ │ mean ┆ 2.266667 ┆ 45.0 ┆ 0.666667 ┆ null ┆ 2021-07-02 16:00:00 ┆ 16:07:10 │ │ std ┆ 1.101514 ┆ 7.071068 ┆ null ┆ null ┆ null ┆ null │ │ min ┆ 1.0 ┆ 40.0 ┆ 0.0 ┆ xx ┆ 2020-01-01 ┆ 10:20:30 │ │ 25% ┆ 2.8 ┆ 40.0 ┆ null ┆ null ┆ 2021-07-05 ┆ 14:45:50 │ │ 50% ┆ 2.8 ┆ 50.0 ┆ null ┆ null ┆ 2021-07-05 ┆ 14:45:50 │ │ 75% ┆ 3.0 ┆ 50.0 ┆ null ┆ null ┆ 2022-12-31 ┆ 23:15:10 │ │ max ┆ 3.0 ┆ 50.0 ┆ 1.0 ┆ zz ┆ 2022-12-31 ┆ 23:15:10 │ └────────────┴──────────┴──────────┴──────────┴──────┴─────────────────────┴──────────┘ Customize which percentiles are displayed, applying linear interpolation: >>> with pl.Config(tbl_rows=12): ... df.describe( ... percentiles=[0.1, 0.3, 0.5, 0.7, 0.9], ... interpolation="linear", ... ) shape: (11, 7) ┌────────────┬──────────┬──────────┬──────────┬──────┬─────────────────────┬──────────┐ │ statistic ┆ float ┆ int ┆ bool ┆ str ┆ date ┆ time │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ f64 ┆ f64 ┆ f64 ┆ str ┆ str ┆ str │ ╞════════════╪══════════╪══════════╪══════════╪══════╪═════════════════════╪══════════╡ │ count ┆ 3.0 ┆ 2.0 ┆ 3.0 ┆ 3 ┆ 3 ┆ 3 │ │ null_count ┆ 0.0 ┆ 1.0 ┆ 0.0 ┆ 0 ┆ 0 ┆ 0 │ │ mean ┆ 2.266667 ┆ 45.0 ┆ 0.666667 ┆ null ┆ 2021-07-02 16:00:00 ┆ 16:07:10 │ │ std ┆ 1.101514 ┆ 7.071068 ┆ null ┆ null ┆ null ┆ null │ │ min ┆ 1.0 ┆ 40.0 ┆ 0.0 ┆ xx ┆ 2020-01-01 ┆ 10:20:30 │ │ 10% ┆ 1.36 ┆ 41.0 ┆ null ┆ null ┆ 2020-04-20 ┆ 11:13:34 │ │ 30% ┆ 2.08 ┆ 43.0 ┆ null ┆ null ┆ 2020-11-26 ┆ 12:59:42 │ │ 50% ┆ 2.8 ┆ 45.0 ┆ null ┆ null ┆ 2021-07-05 ┆ 14:45:50 │ │ 70% ┆ 2.88 ┆ 47.0 ┆ null ┆ null ┆ 2022-02-07 ┆ 18:09:34 │ │ 90% ┆ 2.96 ┆ 49.0 ┆ null ┆ null ┆ 2022-09-13 ┆ 21:33:18 │ │ max ┆ 3.0 ┆ 50.0 ┆ 1.0 ┆ zz ┆ 2022-12-31 ┆ 23:15:10 │ └────────────┴──────────┴──────────┴──────────┴──────┴─────────────────────┴──────────┘ z/cannot describe a DataFrame that has no columns) percentilesr0)rrrdescribe)rr2r0rs rr3DataFrame.describes<z||CCC. yy{###$  rc8URRU5$)aK Find the index of a column by name. Parameters ---------- name Name of the column to find. Examples -------- >>> df = pl.DataFrame( ... {"foo": [1, 2, 3], "bar": [6, 7, 8], "ham": ["a", "b", "c"]} ... ) >>> df.get_column_index("ham") 2 >>> df.get_column_index("sandwich") # doctest: +SKIP ColumnNotFoundError: sandwich )rr(rs rr(DataFrame.get_column_indexs&xx((..rczUS:aURU-nURRXR5 U$)u Replace a column at an index location. This operation is in place. Parameters ---------- index Column index. column Series that will replace the column. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... } ... ) >>> s = pl.Series("apple", [10, 20, 30]) >>> df.replace_column(0, s) shape: (3, 3) ┌───────┬─────┬─────┐ │ apple ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═══════╪═════╪═════╡ │ 10 ┆ 6 ┆ a │ │ 20 ┆ 7 ┆ b │ │ 30 ┆ 8 ┆ c │ └───────┴─────┴─────┘ r)rrrr)rrrs rrDataFrame.replace_columns5F 19JJ&E yy1 r descending nulls_last multithreadedmaintain_orderr`r:cSSKJn UR5R"U/UQ7UUUUS.6R UR 5S9$)u Sort the dataframe by the given columns. Parameters ---------- by Column(s) to sort by. Accepts expression input, including selectors. Strings are parsed as column names. *more_by Additional columns to sort by, specified as positional arguments. descending Sort in descending order. When sorting by multiple columns, can be specified per column by passing a sequence of booleans. nulls_last Place null values last; can specify a single boolean applying to all columns or a sequence of booleans for per-column control. multithreaded Sort using multiple threads. maintain_order Whether the order should be maintained if elements are equal. Examples -------- Pass a single column name to sort by that column. >>> df = pl.DataFrame( ... { ... "a": [1, 2, None], ... "b": [6.0, 5.0, 4.0], ... "c": ["a", "c", "b"], ... } ... ) >>> df.sort("a") shape: (3, 3) ┌──────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str │ ╞══════╪═════╪═════╡ │ null ┆ 4.0 ┆ b │ │ 1 ┆ 6.0 ┆ a │ │ 2 ┆ 5.0 ┆ c │ └──────┴─────┴─────┘ Sorting by expressions is also supported. >>> df.sort(pl.col("a") + pl.col("b") * 2, nulls_last=True) shape: (3, 3) ┌──────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str │ ╞══════╪═════╪═════╡ │ 2 ┆ 5.0 ┆ c │ │ 1 ┆ 6.0 ┆ a │ │ null ┆ 4.0 ┆ b │ └──────┴─────┴─────┘ Sort by multiple columns by passing a list of columns. >>> df.sort(["c", "a"], descending=True) shape: (3, 3) ┌──────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str │ ╞══════╪═════╪═════╡ │ 2 ┆ 5.0 ┆ c │ │ null ┆ 4.0 ┆ b │ │ 1 ┆ 6.0 ┆ a │ └──────┴─────┴─────┘ Or use positional arguments to sort by multiple columns in the same way. >>> df.sort("c", "a", descending=[False, True]) shape: (3, 3) ┌──────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str │ ╞══════╪═════╪═════╡ │ 1 ┆ 6.0 ┆ a │ │ null ┆ 4.0 ┆ b │ │ 2 ┆ 5.0 ┆ c │ └──────┴─────┴─────┘ rrr9r)polars.lazyframerrsortrr)rr`r:r;r<r=more_byrs rr@DataFrame.sort/s^~ 3 IIK T   &%+- W=#7#7#9W : rr)rcSSKJn [S5 U"SSS9nU(aUOSnURXPS9 UR U5sS S S 5 $!,(df  g =f) u~ Execute a SQL query against the DataFrame. .. versionadded:: 0.20.24 .. warning:: This functionality is considered **unstable**, although it is close to being considered stable. It may be changed at any point without it being considered a breaking change. Parameters ---------- query SQL query to execute. table_name Optionally provide an explicit name for the table that represents the calling frame (defaults to "self"). Notes ----- * The calling frame is automatically registered as a table in the SQL context under the name "self". If you want access to the DataFrames and LazyFrames found in the current globals, use the top-level :meth:`pl.sql `. * More control over registration and execution behaviour is available by using the :class:`SQLContext` object. * The SQL query executes in lazy mode before being collected and returned as a DataFrame. See Also -------- SQLContext Examples -------- >>> from datetime import date >>> df1 = pl.DataFrame( ... { ... "a": [1, 2, 3], ... "b": ["zz", "yy", "xx"], ... "c": [date(1999, 12, 31), date(2010, 10, 10), date(2077, 8, 8)], ... } ... ) Query the DataFrame using SQL: >>> df1.sql("SELECT c, b FROM self WHERE a > 1") shape: (2, 2) ┌────────────┬─────┐ │ c ┆ b │ │ --- ┆ --- │ │ date ┆ str │ ╞════════════╪═════╡ │ 2010-10-10 ┆ yy │ │ 2077-08-08 ┆ xx │ └────────────┴─────┘ Apply transformations to a DataFrame using SQL, aliasing "self" to "frame". >>> df1.sql( ... query=''' ... SELECT ... a, ... (a % 2 == 0) AS a_is_even, ... CONCAT_WS(':', b, b) AS b_b, ... EXTRACT(year FROM c) AS year, ... 0::float4 AS "zero", ... FROM frame ... ''', ... table_name="frame", ... ) shape: (3, 5) ┌─────┬───────────┬───────┬──────┬──────┐ │ a ┆ a_is_even ┆ b_b ┆ year ┆ zero │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ bool ┆ str ┆ i32 ┆ f32 │ ╞═════╪═══════════╪═══════╪══════╪══════╡ │ 1 ┆ false ┆ zz:zz ┆ 1999 ┆ 0.0 │ │ 2 ┆ true ┆ yy:yy ┆ 2010 ┆ 0.0 │ │ 3 ┆ false ┆ xx:xx ┆ 2077 ┆ 0.0 │ └─────┴───────────┴───────┴──────┴──────┘ r) SQLContextzS`sql` is considered **unstable** (although it is close to being considered stable).FT)register_globalseagerr)rreN) polars.sqlrDr-registerr)rqueryrrDctxrs rsql DataFrame.sqlsPd * a d ;s!+:D LLdL /;;u%< ; ;s +A Arz1.0.0)rc rSSKJn UR5RXUS9R U"SSSSS9S9$)uS Return the `k` largest rows. Non-null elements are always preferred over null elements, regardless of the value of `reverse`. The output is not guaranteed to be in any particular order, call :func:`sort` after this function if you wish the output to be sorted. .. versionchanged:: 1.0.0 The `descending` parameter was renamed `reverse`. Parameters ---------- k Number of rows to return. by Column(s) used to determine the top rows. Accepts expression input. Strings are parsed as column names. reverse Consider the `k` smallest elements of the `by` column(s) (instead of the `k` largest). This can be specified per column by passing a sequence of booleans. See Also -------- bottom_k Examples -------- >>> df = pl.DataFrame( ... { ... "a": ["a", "b", "a", "b", "b", "c"], ... "b": [2, 1, 1, 3, 2, 1], ... } ... ) Get the rows which contain the 4 largest values in column b. >>> df.top_k(4, by="b") shape: (4, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═════╪═════╡ │ b ┆ 3 │ │ a ┆ 2 │ │ b ┆ 2 │ │ b ┆ 1 │ └─────┴─────┘ Get the rows which contain the 4 largest values when sorting on column b and a. >>> df.top_k(4, by=["b", "a"]) shape: (4, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═════╪═════╡ │ b ┆ 3 │ │ b ┆ 2 │ │ a ┆ 2 │ │ c ┆ 1 │ └─────┴─────┘ rrr`rFTprojection_pushdownpredicate_pushdowncomm_subplan_elimslice_pushdownr)rrrtop_krrkr`rrs rrTDataFrame.top_ksIT = IIK U1WU - W+(-',&+#'  rc rSSKJn UR5RXUS9R U"SSSSS9S9$)u\ Return the `k` smallest rows. Non-null elements are always preferred over null elements, regardless of the value of `reverse`. The output is not guaranteed to be in any particular order, call :func:`sort` after this function if you wish the output to be sorted. .. versionchanged:: 1.0.0 The `descending` parameter was renamed `reverse`. Parameters ---------- k Number of rows to return. by Column(s) used to determine the bottom rows. Accepts expression input. Strings are parsed as column names. reverse Consider the `k` largest elements of the `by` column(s) (instead of the `k` smallest). This can be specified per column by passing a sequence of booleans. See Also -------- top_k Examples -------- >>> df = pl.DataFrame( ... { ... "a": ["a", "b", "a", "b", "b", "c"], ... "b": [2, 1, 1, 3, 2, 1], ... } ... ) Get the rows which contain the 4 smallest values in column b. >>> df.bottom_k(4, by="b") shape: (4, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═════╪═════╡ │ b ┆ 1 │ │ a ┆ 1 │ │ c ┆ 1 │ │ a ┆ 2 │ └─────┴─────┘ Get the rows which contain the 4 smallest values when sorting on column a and b. >>> df.bottom_k(4, by=["a", "b"]) shape: (4, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═════╪═════╡ │ a ┆ 1 │ │ a ┆ 2 │ │ b ┆ 1 │ │ b ┆ 2 │ └─────┴─────┘ rrrNFTrOr)rrrbottom_krrUs rrYDataFrame.bottom_kRsIT = IIK XaX 0 W+(-',&+#'  r null_equalc`[X5 URRURUS9$)aQ Check whether the DataFrame is equal to another DataFrame. Parameters ---------- other DataFrame to compare with. null_equal Consider null values as equal. See Also -------- polars.testing.assert_frame_equal Examples -------- >>> df1 = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6.0, 7.0, 8.0], ... "ham": ["a", "b", "c"], ... } ... ) >>> df2 = pl.DataFrame( ... { ... "foo": [3, 2, 1], ... "bar": [8.0, 7.0, 6.0], ... "ham": ["c", "b", "a"], ... } ... ) >>> df1.equals(df1) True >>> df1.equals(df2) False r[)r4requals)rr:r\s rr^DataFrame.equalss(H $&xxuyyZ@@rcUbUS:aURU- U-nURURRX55$)u Get a slice of this DataFrame. Parameters ---------- offset Start index. Negative indexing is supported. length Length of the slice. If set to `None`, all rows starting at the offset will be selected. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6.0, 7.0, 8.0], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.slice(1, 2) shape: (2, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str │ ╞═════╪═════╪═════╡ │ 2 ┆ 7.0 ┆ b │ │ 3 ┆ 8.0 ┆ c │ └─────┴─────┴─────┘ r)rrrslice)roffsetrWs rraDataFrame.slices@@  FQJ[[6)F2Ftxx~~f=>>rcUS:a[SURU-5nURURR U55$)u Get the first `n` rows. Parameters ---------- n Number of rows to return. If a negative value is passed, return all rows except the last `abs(n)`. See Also -------- tail, glimpse, slice Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3, 4, 5], ... "bar": [6, 7, 8, 9, 10], ... "ham": ["a", "b", "c", "d", "e"], ... } ... ) >>> df.head(3) shape: (3, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ │ 2 ┆ 7 ┆ b │ │ 3 ┆ 8 ┆ c │ └─────┴─────┴─────┘ Pass a negative value to get all rows `except` the last `abs(n)`. >>> df.head(-3) shape: (2, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ │ 2 ┆ 7 ┆ b │ └─────┴─────┴─────┘ r)r+rrrheadrrQs rreDataFrame.head<` q5At{{Q'Atxx}}Q/00rcUS:a[SURU-5nURURR U55$)u Get the last `n` rows. Parameters ---------- n Number of rows to return. If a negative value is passed, return all rows except the first `abs(n)`. See Also -------- head, slice Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3, 4, 5], ... "bar": [6, 7, 8, 9, 10], ... "ham": ["a", "b", "c", "d", "e"], ... } ... ) >>> df.tail(3) shape: (3, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 3 ┆ 8 ┆ c │ │ 4 ┆ 9 ┆ d │ │ 5 ┆ 10 ┆ e │ └─────┴─────┴─────┘ Pass a negative value to get all rows `except` the first `abs(n)`. >>> df.tail(-3) shape: (2, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 4 ┆ 9 ┆ d │ │ 5 ┆ 10 ┆ e │ └─────┴─────┴─────┘ r)r+rrrtailrfs rrjDataFrame.tail*rhrc$URU5$)u- Get the first `n` rows. Alias for :func:`DataFrame.head`. Parameters ---------- n Number of rows to return. If a negative value is passed, return all rows except the last `abs(n)`. See Also -------- head Examples -------- Get the first 3 rows of a DataFrame. >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3, 4, 5], ... "bar": [6, 7, 8, 9, 10], ... "ham": ["a", "b", "c", "d", "e"], ... } ... ) >>> df.limit(3) shape: (3, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ │ 2 ┆ 7 ┆ b │ │ 3 ┆ 8 ┆ c │ └─────┴─────┴─────┘ )rerfs rlimitDataFrame.limit^sNyy|rcSSKJn UR5RU5R UR 5S9$)u Drop all rows that contain one or more NaN values. The original order of the remaining rows is preserved. Parameters ---------- subset Column name(s) for which NaN values are considered; if set to `None` (default), use all columns (note that only floating-point columns can contain NaNs). See Also -------- drop_nulls Notes ----- A NaN value is not the same as a null value. To drop null values, use :func:`drop_nulls`. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [-20.5, float("nan"), 80.0], ... "bar": [float("nan"), 110.0, 25.5], ... "ham": ["xxx", "yyy", None], ... } ... ) The default behavior of this method is to drop rows where any single value in the row is NaN: >>> df.drop_nans() shape: (1, 3) ┌──────┬──────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ str │ ╞══════╪══════╪══════╡ │ 80.0 ┆ 25.5 ┆ null │ └──────┴──────┴──────┘ This behaviour can be constrained to consider only a subset of columns, as defined by name, or with a selector. For example, dropping rows only if there is a NaN in the "bar" column: >>> df.drop_nans(subset=["bar"]) shape: (2, 3) ┌──────┬───────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ str │ ╞══════╪═══════╪══════╡ │ NaN ┆ 110.0 ┆ yyy │ │ 80.0 ┆ 25.5 ┆ null │ └──────┴───────┴──────┘ Dropping a row only if *all* values are NaN requires a different formulation: >>> df = pl.DataFrame( ... { ... "a": [float("nan"), float("nan"), float("nan"), float("nan")], ... "b": [10.0, 2.5, float("nan"), 5.25], ... "c": [65.75, float("nan"), float("nan"), 10.5], ... } ... ) >>> df.filter(~pl.all_horizontal(pl.all().is_nan())) shape: (3, 3) ┌─────┬──────┬───────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ f64 │ ╞═════╪══════╪═══════╡ │ NaN ┆ 10.0 ┆ 65.75 │ │ NaN ┆ 2.5 ┆ NaN │ │ NaN ┆ 5.25 ┆ 10.5 │ └─────┴──────┴───────┘ rrr)rrr drop_nansrrrsubsetrs rrpDataFrame.drop_nanss:h = IIK ! !& ) 1 1 @T@T@V 1 W rcSSKJn UR5RU5R UR 5S9$)uW Drop all rows that contain one or more null values. The original order of the remaining rows is preserved. Parameters ---------- subset Column name(s) for which null values are considered. If set to `None` (default), use all columns. See Also -------- drop_nans Notes ----- A null value is not the same as a NaN value. To drop NaN values, use :func:`drop_nans`. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, None, 8], ... "ham": ["a", "b", None], ... } ... ) The default behavior of this method is to drop rows where any single value of the row is null. >>> df.drop_nulls() shape: (1, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ └─────┴─────┴─────┘ This behaviour can be constrained to consider only a subset of columns, as defined by name or with a selector. For example, dropping rows if there is a null in any of the integer columns: >>> import polars.selectors as cs >>> df.drop_nulls(subset=cs.integer()) shape: (2, 3) ┌─────┬─────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪══════╡ │ 1 ┆ 6 ┆ a │ │ 3 ┆ 8 ┆ null │ └─────┴─────┴──────┘ Below are some additional examples that show how to drop null values based on other conditions. >>> df = pl.DataFrame( ... { ... "a": [None, None, None, None], ... "b": [1, 2, None, 1], ... "c": [1, None, None, 1], ... } ... ) >>> df shape: (4, 3) ┌──────┬──────┬──────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ null ┆ i64 ┆ i64 │ ╞══════╪══════╪══════╡ │ null ┆ 1 ┆ 1 │ │ null ┆ 2 ┆ null │ │ null ┆ null ┆ null │ │ null ┆ 1 ┆ 1 │ └──────┴──────┴──────┘ Drop a row only if all values are null: >>> df.filter(~pl.all_horizontal(pl.all().is_null())) shape: (3, 3) ┌──────┬─────┬──────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ null ┆ i64 ┆ i64 │ ╞══════╪═════╪══════╡ │ null ┆ 1 ┆ 1 │ │ null ┆ 2 ┆ null │ │ null ┆ 1 ┆ 1 │ └──────┴─────┴──────┘ Drop a column if all values are null: >>> df[[s.name for s in df if not (s.null_count() == df.height)]] shape: (4, 2) ┌──────┬──────┐ │ b ┆ c │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞══════╪══════╡ │ 1 ┆ 1 │ │ 2 ┆ null │ │ null ┆ null │ │ 1 ┆ 1 │ └──────┴──────┘ rrr)rrr drop_nullsrrrqs rruDataFrame.drop_nullss:f = IIK " "6 * 2 2AUAUAW 2 X rcU"U/UQ70UD6$)u= Offers a structured way to apply a sequence of user-defined functions (UDFs). Parameters ---------- function Callable; will receive the frame as the first parameter, followed by any given args/kwargs. *args Arguments to pass to the UDF. **kwargs Keyword arguments to pass to the UDF. Notes ----- It is recommended to use LazyFrame when piping operations, in order to fully take advantage of query optimization and parallelization. See :meth:`df.lazy() `. Examples -------- >>> def cast_str_to_int(data, col_name): ... return data.with_columns(pl.col(col_name).cast(pl.Int64)) >>> df = pl.DataFrame({"a": [1, 2, 3, 4], "b": ["10", "20", "30", "40"]}) >>> df.pipe(cast_str_to_int, col_name="b") shape: (4, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 10 │ │ 2 ┆ 20 │ │ 3 ┆ 30 │ │ 4 ┆ 40 │ └─────┴─────┘ >>> df = pl.DataFrame({"b": [1, 2], "a": [3, 4]}) >>> df shape: (2, 2) ┌─────┬─────┐ │ b ┆ a │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 3 │ │ 2 ┆ 4 │ └─────┴─────┘ >>> df.pipe(lambda tdf: tdf.select(sorted(tdf.columns))) shape: (2, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 3 ┆ 1 │ │ 4 ┆ 2 │ └─────┴─────┘ r)rfunctionargsrs rpipeDataFrame.pipeZsB.t.v..rc @[U[R[R45(aSSKJn [ U"X55nO$[U[5(aU/nO [ U5nUR"S0UVs0sHowU"X/UQ70UD6_M snD6$s snf)u Apply eager functions to columns of a DataFrame. Users should always prefer :meth:`with_columns` unless they are using expressions that are only possible on `Series` and not on `Expr`. This is almost never the case, except for a very select few functions that cannot know the output datatype without looking at the data. Parameters ---------- column_names The columns to apply the UDF to. function Callable; will receive a column series as the first parameter, followed by any given args/kwargs. *args Arguments to pass to the UDF. **kwargs Keyword arguments to pass to the UDF. Examples -------- >>> df = pl.DataFrame({"a": [1, 2, 3, 4], "b": ["10", "20", "30", "40"]}) >>> df.map_columns("a", lambda s: s.shrink_dtype()) shape: (4, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i8 ┆ str │ ╞═════╪═════╡ │ 1 ┆ 10 │ │ 2 ┆ 20 │ │ 3 ┆ 30 │ │ 4 ┆ 40 │ └─────┴─────┘ >>> df = pl.DataFrame( ... { ... "a": ['{"x":"a"}', None, '{"x":"b"}', None], ... "b": ['{"a":1, "b": true}', None, '{"a":2, "b": false}', None], ... } ... ) >>> df.map_columns(["a", "b"], lambda s: s.str.json_decode()) shape: (4, 2) ┌───────────┬───────────┐ │ a ┆ b │ │ --- ┆ --- │ │ struct[1] ┆ struct[2] │ ╞═══════════╪═══════════╡ │ {"a"} ┆ {1,true} │ │ null ┆ null │ │ {"b"} ┆ {2,false} │ │ null ┆ null │ └───────────┴───────────┘ >>> import polars.selectors as cs >>> df.map_columns(cs.all(), lambda s: s.str.json_decode()) shape: (4, 2) ┌───────────┬───────────┐ │ a ┆ b │ │ --- ┆ --- │ │ struct[1] ┆ struct[2] │ ╞═══════════╪═══════════╡ │ {"a"} ┆ {1,true} │ │ null ┆ null │ │ {"b"} ┆ {2,false} │ │ null ┆ null │ └───────────┴───────────┘ See Also -------- with_columns r)expand_selectorr) rrSelectorrspolars.selectorsr}rrr])rrrxryrr}c_namesr?s r map_columnsDataFrame.map_columnss` lR[["''$: ; ; 8?4>?G  c * *#nG<(G   >EFg(474T4V44gF  Fs>B cURURRX55$![a US:aSOSnSUSU3n[ U5Sef=f)u= Add a row index as the first column in the DataFrame. Parameters ---------- name Name of the index column. offset Start the index at this offset. Cannot be negative. Notes ----- The resulting column does not have any special properties. It is a regular column of type `UInt32` (or `UInt64` in `polars-u64-idx`). Examples -------- >>> df = pl.DataFrame( ... { ... "a": [1, 3, 5], ... "b": [2, 4, 6], ... } ... ) >>> df.with_row_index() shape: (3, 3) ┌───────┬─────┬─────┐ │ index ┆ a ┆ b │ │ --- ┆ --- ┆ --- │ │ u32 ┆ i64 ┆ i64 │ ╞═══════╪═════╪═════╡ │ 0 ┆ 1 ┆ 2 │ │ 1 ┆ 3 ┆ 4 │ │ 2 ┆ 5 ┆ 6 │ └───────┴─────┴─────┘ >>> df.with_row_index("id", offset=1000) shape: (3, 3) ┌──────┬─────┬─────┐ │ id ┆ a ┆ b │ │ --- ┆ --- ┆ --- │ │ u32 ┆ i64 ┆ i64 │ ╞══════╪═════╪═════╡ │ 1000 ┆ 1 ┆ 2 │ │ 1001 ┆ 3 ┆ 4 │ │ 1002 ┆ 5 ┆ 6 │ └──────┴─────┴─────┘ An index column can also be created using the expressions :func:`int_range` and :func:`len`. >>> df.select( ... pl.int_range(pl.len(), dtype=pl.UInt32).alias("index"), ... pl.all(), ... ) shape: (3, 3) ┌───────┬─────┬─────┐ │ index ┆ a ┆ b │ │ --- ┆ --- ┆ --- │ │ u32 ┆ i64 ┆ i64 │ ╞═══════╪═════╪═════╡ │ 0 ┆ 1 ┆ 2 │ │ 1 ┆ 3 ┆ 4 │ │ 2 ┆ 5 ┆ 6 │ └───────┴─────┴─────┘ rnegativez$greater than the maximum index valuez.`offset` input for `with_row_index` cannot be z, got N)rrwith_row_index OverflowErrorr)rrrbissuers rrDataFrame.with_row_indexsbB ,??488#:#:4#HI I ,"(1*J2XEB5'PVxXCS/t + ,s ),)Az`DataFrame.with_row_count` is deprecated; use `with_row_index` instead. Note that the default column name has changed from 'row_nr' to 'index'.c$URX5$)us Add a column at index 0 that counts the rows. .. deprecated:: 0.20.4 Use the :meth:`with_row_index` method instead. Note that the default column name has changed from 'row_nr' to 'index'. Parameters ---------- name Name of the column to add. offset Start the row count at this offset. Default = 0 Examples -------- >>> df = pl.DataFrame( ... { ... "a": [1, 3, 5], ... "b": [2, 4, 6], ... } ... ) >>> df.with_row_count() # doctest: +SKIP shape: (3, 3) ┌────────┬─────┬─────┐ │ row_nr ┆ a ┆ b │ │ --- ┆ --- ┆ --- │ │ u32 ┆ i64 ┆ i64 │ ╞════════╪═════╪═════╡ │ 0 ┆ 1 ┆ 2 │ │ 1 ┆ 3 ┆ 4 │ │ 2 ┆ 5 ┆ 6 │ └────────┴─────┴─────┘ )r)rrrbs rwith_row_countDataFrame.with_row_countBsN""400rr=cUR5HYn[U[[R[R 45(aM9S[ U5SU<SU<S3n[U5e [U/UQ70UDSU0D6$)u Start a group by operation. Parameters ---------- *by Column(s) to group by. Accepts expression input. Strings are parsed as column names. maintain_order Ensure that the order of the groups is consistent with the input data. This is slower than a default group by. Settings this to `True` blocks the possibility to run on the streaming engine. .. note:: Within each group, the order of rows is always preserved, regardless of this argument. **named_by Additional columns to group by, specified as keyword arguments. The columns will be renamed to the keyword used. Returns ------- GroupBy Object which can be used to perform aggregations. Examples -------- Group by one column and call `agg` to compute the grouped sum of another column. >>> df = pl.DataFrame( ... { ... "a": ["a", "b", "a", "b", "c"], ... "b": [1, 2, 1, 3, 3], ... "c": [5, 4, 3, 2, 1], ... } ... ) >>> df.group_by("a").agg(pl.col("b").sum()) # doctest: +IGNORE_RESULT shape: (3, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═════╪═════╡ │ a ┆ 2 │ │ b ┆ 5 │ │ c ┆ 3 │ └─────┴─────┘ Set `maintain_order=True` to ensure the order of the groups is consistent with the input. >>> df.group_by("a", maintain_order=True).agg(pl.col("c")) shape: (3, 2) ┌─────┬───────────┐ │ a ┆ c │ │ --- ┆ --- │ │ str ┆ list[i64] │ ╞═════╪═══════════╡ │ a ┆ [5, 3] │ │ b ┆ [4, 2] │ │ c ┆ [1] │ └─────┴───────────┘ Group by multiple columns by passing a list of column names. >>> df.group_by(["a", "b"]).agg(pl.max("c")) # doctest: +IGNORE_RESULT shape: (4, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ a ┆ 1 ┆ 5 │ │ b ┆ 2 ┆ 4 │ │ b ┆ 3 ┆ 2 │ │ c ┆ 3 ┆ 1 │ └─────┴─────┴─────┘ Or use positional arguments to group by multiple columns in the same way. Expressions are also accepted. >>> df.group_by("a", pl.col("b") // 2).agg(pl.col("c").mean()) # doctest: +SKIP shape: (3, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ f64 │ ╞═════╪═════╪═════╡ │ a ┆ 0 ┆ 4.0 │ │ b ┆ 1 ┆ 3.0 │ │ c ┆ 1 ┆ 1.0 │ └─────┴─────┴─────┘ The `GroupBy` object returned by this method is iterable, returning the name and data of each group. >>> for name, data in df.group_by("a"): # doctest: +SKIP ... print(name) ... print(data) ('a',) shape: (2, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ a ┆ 1 ┆ 5 │ │ a ┆ 1 ┆ 3 │ └─────┴─────┴─────┘ ('b',) shape: (2, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ b ┆ 2 ┆ 4 │ │ b ┆ 3 ┆ 2 │ └─────┴─────┴─────┘ ('c',) shape: (1, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ c ┆ 3 ┆ 1 │ └─────┴─────┴─────┘ z=Expected Polars expression or object convertible to one, got z&. Hint: if you tried group_by(by=z;) then you probably want to use this instead: group_by(rr=) rrrrrsrurrr<)rr=r`named_byrrs rgroup_byDataFrame.group_byksR__&Eec277BII%>??STXY^T_S`a'',i0$$)9A / n$'tLbLHL^LLrrrright)rbclosedrc [UUUUUUS9$)ua Create rolling groups based on a temporal or integer column. Different from a `group_by_dynamic` the windows are now determined by the individual values and are not of constant intervals. For constant intervals use :func:`DataFrame.group_by_dynamic`. If you have a time series ``, then by default the windows created will be * (t_0 - period, t_0] * (t_1 - period, t_1] * ... * (t_n - period, t_n] whereas if you pass a non-default `offset`, then the windows will be * (t_0 + offset, t_0 + offset + period] * (t_1 + offset, t_1 + offset + period] * ... * (t_n + offset, t_n + offset + period] The `period` and `offset` arguments are created either from a timedelta, or by using the following string language: - 1ns (1 nanosecond) - 1us (1 microsecond) - 1ms (1 millisecond) - 1s (1 second) - 1m (1 minute) - 1h (1 hour) - 1d (1 calendar day) - 1w (1 calendar week) - 1mo (1 calendar month) - 1q (1 calendar quarter) - 1y (1 calendar year) - 1i (1 index count) Or combine them: "3d12h4m25s" # 3 days, 12 hours, 4 minutes, and 25 seconds By "calendar day", we mean the corresponding time on the next day (which may not be 24 hours, due to daylight savings). Similarly for "calendar week", "calendar month", "calendar quarter", and "calendar year". .. versionchanged:: 0.20.14 The `by` parameter was renamed `group_by`. Parameters ---------- index_column Column used to group based on the time window. Often of type Date/Datetime. This column must be sorted in ascending order (or, if `group_by` is specified, then it must be sorted in ascending order within each group). In case of a rolling operation on indices, dtype needs to be one of {UInt32, UInt64, Int32, Int64}. Note that the first three get temporarily cast to Int64, so if performance matters use an Int64 column. period Length of the window - must be non-negative. offset Offset of the window. Default is `-period`. closed : {'right', 'left', 'both', 'none'} Define which sides of the temporal interval are closed (inclusive). group_by Also group by this column/these columns Returns ------- RollingGroupBy Object you can call `.agg` on to aggregate by groups, the result of which will be sorted by `index_column` (but note that if `group_by` columns are passed, it will only be sorted within each group). See Also -------- group_by_dynamic Examples -------- >>> dates = [ ... "2020-01-01 13:45:48", ... "2020-01-01 16:42:13", ... "2020-01-01 16:45:09", ... "2020-01-02 18:12:48", ... "2020-01-03 19:45:32", ... "2020-01-08 23:16:43", ... ] >>> df = pl.DataFrame({"dt": dates, "a": [3, 7, 5, 9, 2, 1]}).with_columns( ... pl.col("dt").str.strptime(pl.Datetime).set_sorted() ... ) >>> out = df.rolling(index_column="dt", period="2d").agg( ... [ ... pl.sum("a").alias("sum_a"), ... pl.min("a").alias("min_a"), ... pl.max("a").alias("max_a"), ... ] ... ) >>> assert out["sum_a"].to_list() == [3, 10, 15, 24, 11, 1] >>> assert out["max_a"].to_list() == [3, 7, 7, 9, 9, 1] >>> assert out["min_a"].to_list() == [3, 3, 3, 3, 2, 1] >>> out shape: (6, 4) ┌─────────────────────┬───────┬───────┬───────┐ │ dt ┆ sum_a ┆ min_a ┆ max_a │ │ --- ┆ --- ┆ --- ┆ --- │ │ datetime[μs] ┆ i64 ┆ i64 ┆ i64 │ ╞═════════════════════╪═══════╪═══════╪═══════╡ │ 2020-01-01 13:45:48 ┆ 3 ┆ 3 ┆ 3 │ │ 2020-01-01 16:42:13 ┆ 10 ┆ 3 ┆ 7 │ │ 2020-01-01 16:45:09 ┆ 15 ┆ 3 ┆ 7 │ │ 2020-01-02 18:12:48 ┆ 24 ┆ 3 ┆ 9 │ │ 2020-01-03 19:45:32 ┆ 11 ┆ 2 ┆ 9 │ │ 2020-01-08 23:16:43 ┆ 1 ┆ 1 ┆ 1 │ └─────────────────────┴───────┴───────┴───────┘ If you use an index count in `period` or `offset`, then it's based on the values in `index_column`: >>> df = pl.DataFrame({"int": [0, 4, 5, 6, 8], "value": [1, 4, 2, 4, 1]}) >>> df.rolling("int", period="3i").agg(pl.col("int").alias("aggregated")) shape: (5, 2) ┌─────┬────────────┐ │ int ┆ aggregated │ │ --- ┆ --- │ │ i64 ┆ list[i64] │ ╞═════╪════════════╡ │ 0 ┆ [0] │ │ 4 ┆ [4] │ │ 5 ┆ [4, 5] │ │ 6 ┆ [4, 5, 6] │ │ 8 ┆ [6, 8] │ └─────┴────────────┘ If you want the index count to be based on row number, then you may want to combine `rolling` with :meth:`.with_row_index`. ) index_columnperiodrbrr)r=)rrrrbrrs rrollingDataFrame.rollings$h %   rleftwindow)rrbinclude_boundariesrrBrstart_byc &[UUUUUUUUUU S9 $)u5 Group based on a time value (or index value of type Int32, Int64). Time windows are calculated and rows are assigned to windows. Different from a normal group by is that a row can be member of multiple groups. By default, the windows look like: - [start, start + period) - [start + every, start + every + period) - [start + 2*every, start + 2*every + period) - ... where `start` is determined by `start_by`, `offset`, `every`, and the earliest datapoint. See the `start_by` argument description for details. .. warning:: The index column must be sorted in ascending order. If `group_by` is passed, then the index column must be sorted in ascending order within each group. .. versionchanged:: 0.20.14 The `by` parameter was renamed `group_by`. Parameters ---------- index_column Column used to group based on the time window. Often of type Date/Datetime. This column must be sorted in ascending order (or, if `group_by` is specified, then it must be sorted in ascending order within each group). In case of a dynamic group by on indices, dtype needs to be one of {Int32, Int64}. Note that Int32 gets temporarily cast to Int64, so if performance matters use an Int64 column. every interval of the window period length of the window, if None it will equal 'every' offset offset of the window, does not take effect if `start_by` is 'datapoint'. Defaults to zero. include_boundaries Add the lower and upper bound of the window to the "_lower_boundary" and "_upper_boundary" columns. This will impact performance because it's harder to parallelize closed : {'left', 'right', 'both', 'none'} Define which sides of the temporal interval are closed (inclusive). label : {'left', 'right', 'datapoint'} Define which label to use for the window: - 'left': lower boundary of the window - 'right': upper boundary of the window - 'datapoint': the first value of the index column in the given window. If you don't need the label to be at one of the boundaries, choose this option for maximum performance group_by Also group by this column/these columns start_by : {'window', 'datapoint', 'monday', 'tuesday', 'wednesday', 'thursday', 'friday', 'saturday', 'sunday'} The strategy to determine the start of the first window by. * 'window': Start by taking the earliest timestamp, truncating it with `every`, and then adding `offset`. Note that weekly windows start on Monday. * 'datapoint': Start from the first encountered data point. * a day of the week (only takes effect if `every` contains `'w'`): * 'monday': Start the window on the Monday before the first data point. * 'tuesday': Start the window on the Tuesday before the first data point. * ... * 'sunday': Start the window on the Sunday before the first data point. The resulting window is then shifted back until the earliest datapoint is in or in front of it. Returns ------- DynamicGroupBy Object you can call `.agg` on to aggregate by groups, the result of which will be sorted by `index_column` (but note that if `group_by` columns are passed, it will only be sorted within each group). See Also -------- rolling Notes ----- 1) If you're coming from pandas, then .. code-block:: python # polars df.group_by_dynamic("ts", every="1d").agg(pl.col("value").sum()) is equivalent to .. code-block:: python # pandas df.set_index("ts").resample("D")["value"].sum().reset_index() though note that, unlike pandas, polars doesn't add extra rows for empty windows. If you need `index_column` to be evenly spaced, then please combine with :func:`DataFrame.upsample`. 2) The `every`, `period` and `offset` arguments are created with the following string language: - 1ns (1 nanosecond) - 1us (1 microsecond) - 1ms (1 millisecond) - 1s (1 second) - 1m (1 minute) - 1h (1 hour) - 1d (1 calendar day) - 1w (1 calendar week) - 1mo (1 calendar month) - 1q (1 calendar quarter) - 1y (1 calendar year) - 1i (1 index count) Or combine them (except in `every`): "3d12h4m25s" # 3 days, 12 hours, 4 minutes, and 25 seconds By "calendar day", we mean the corresponding time on the next day (which may not be 24 hours, due to daylight savings). Similarly for "calendar week", "calendar month", "calendar quarter", and "calendar year". In case of a group_by_dynamic on an integer column, the windows are defined by: - "1i" # length 1 - "10i" # length 10 Examples -------- >>> from datetime import datetime >>> df = pl.DataFrame( ... { ... "time": pl.datetime_range( ... start=datetime(2021, 12, 16), ... end=datetime(2021, 12, 16, 3), ... interval="30m", ... eager=True, ... ), ... "n": range(7), ... } ... ) >>> df shape: (7, 2) ┌─────────────────────┬─────┐ │ time ┆ n │ │ --- ┆ --- │ │ datetime[μs] ┆ i64 │ ╞═════════════════════╪═════╡ │ 2021-12-16 00:00:00 ┆ 0 │ │ 2021-12-16 00:30:00 ┆ 1 │ │ 2021-12-16 01:00:00 ┆ 2 │ │ 2021-12-16 01:30:00 ┆ 3 │ │ 2021-12-16 02:00:00 ┆ 4 │ │ 2021-12-16 02:30:00 ┆ 5 │ │ 2021-12-16 03:00:00 ┆ 6 │ └─────────────────────┴─────┘ Group by windows of 1 hour. >>> df.group_by_dynamic("time", every="1h", closed="right").agg(pl.col("n")) shape: (4, 2) ┌─────────────────────┬───────────┐ │ time ┆ n │ │ --- ┆ --- │ │ datetime[μs] ┆ list[i64] │ ╞═════════════════════╪═══════════╡ │ 2021-12-15 23:00:00 ┆ [0] │ │ 2021-12-16 00:00:00 ┆ [1, 2] │ │ 2021-12-16 01:00:00 ┆ [3, 4] │ │ 2021-12-16 02:00:00 ┆ [5, 6] │ └─────────────────────┴───────────┘ The window boundaries can also be added to the aggregation result >>> df.group_by_dynamic( ... "time", every="1h", include_boundaries=True, closed="right" ... ).agg(pl.col("n").mean()) shape: (4, 4) ┌─────────────────────┬─────────────────────┬─────────────────────┬─────┐ │ _lower_boundary ┆ _upper_boundary ┆ time ┆ n │ │ --- ┆ --- ┆ --- ┆ --- │ │ datetime[μs] ┆ datetime[μs] ┆ datetime[μs] ┆ f64 │ ╞═════════════════════╪═════════════════════╪═════════════════════╪═════╡ │ 2021-12-15 23:00:00 ┆ 2021-12-16 00:00:00 ┆ 2021-12-15 23:00:00 ┆ 0.0 │ │ 2021-12-16 00:00:00 ┆ 2021-12-16 01:00:00 ┆ 2021-12-16 00:00:00 ┆ 1.5 │ │ 2021-12-16 01:00:00 ┆ 2021-12-16 02:00:00 ┆ 2021-12-16 01:00:00 ┆ 3.5 │ │ 2021-12-16 02:00:00 ┆ 2021-12-16 03:00:00 ┆ 2021-12-16 02:00:00 ┆ 5.5 │ └─────────────────────┴─────────────────────┴─────────────────────┴─────┘ When closed="left", the window excludes the right end of interval: [lower_bound, upper_bound) >>> df.group_by_dynamic("time", every="1h", closed="left").agg(pl.col("n")) shape: (4, 2) ┌─────────────────────┬───────────┐ │ time ┆ n │ │ --- ┆ --- │ │ datetime[μs] ┆ list[i64] │ ╞═════════════════════╪═══════════╡ │ 2021-12-16 00:00:00 ┆ [0, 1] │ │ 2021-12-16 01:00:00 ┆ [2, 3] │ │ 2021-12-16 02:00:00 ┆ [4, 5] │ │ 2021-12-16 03:00:00 ┆ [6] │ └─────────────────────┴───────────┘ When closed="both" the time values at the window boundaries belong to 2 groups. >>> df.group_by_dynamic("time", every="1h", closed="both").agg(pl.col("n")) shape: (4, 2) ┌─────────────────────┬───────────┐ │ time ┆ n │ │ --- ┆ --- │ │ datetime[μs] ┆ list[i64] │ ╞═════════════════════╪═══════════╡ │ 2021-12-16 00:00:00 ┆ [0, 1, 2] │ │ 2021-12-16 01:00:00 ┆ [2, 3, 4] │ │ 2021-12-16 02:00:00 ┆ [4, 5, 6] │ │ 2021-12-16 03:00:00 ┆ [6] │ └─────────────────────┴───────────┘ Dynamic group bys can also be combined with grouping on normal keys >>> df = df.with_columns(groups=pl.Series(["a", "a", "a", "b", "b", "a", "a"])) >>> df shape: (7, 3) ┌─────────────────────┬─────┬────────┐ │ time ┆ n ┆ groups │ │ --- ┆ --- ┆ --- │ │ datetime[μs] ┆ i64 ┆ str │ ╞═════════════════════╪═════╪════════╡ │ 2021-12-16 00:00:00 ┆ 0 ┆ a │ │ 2021-12-16 00:30:00 ┆ 1 ┆ a │ │ 2021-12-16 01:00:00 ┆ 2 ┆ a │ │ 2021-12-16 01:30:00 ┆ 3 ┆ b │ │ 2021-12-16 02:00:00 ┆ 4 ┆ b │ │ 2021-12-16 02:30:00 ┆ 5 ┆ a │ │ 2021-12-16 03:00:00 ┆ 6 ┆ a │ └─────────────────────┴─────┴────────┘ >>> df.group_by_dynamic( ... "time", ... every="1h", ... closed="both", ... group_by="groups", ... include_boundaries=True, ... ).agg(pl.col("n")) shape: (6, 5) ┌────────┬─────────────────────┬─────────────────────┬─────────────────────┬───────────┐ │ groups ┆ _lower_boundary ┆ _upper_boundary ┆ time ┆ n │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ datetime[μs] ┆ datetime[μs] ┆ datetime[μs] ┆ list[i64] │ ╞════════╪═════════════════════╪═════════════════════╪═════════════════════╪═══════════╡ │ a ┆ 2021-12-16 00:00:00 ┆ 2021-12-16 01:00:00 ┆ 2021-12-16 00:00:00 ┆ [0, 1, 2] │ │ a ┆ 2021-12-16 01:00:00 ┆ 2021-12-16 02:00:00 ┆ 2021-12-16 01:00:00 ┆ [2] │ │ a ┆ 2021-12-16 02:00:00 ┆ 2021-12-16 03:00:00 ┆ 2021-12-16 02:00:00 ┆ [5, 6] │ │ a ┆ 2021-12-16 03:00:00 ┆ 2021-12-16 04:00:00 ┆ 2021-12-16 03:00:00 ┆ [6] │ │ b ┆ 2021-12-16 01:00:00 ┆ 2021-12-16 02:00:00 ┆ 2021-12-16 01:00:00 ┆ [3, 4] │ │ b ┆ 2021-12-16 02:00:00 ┆ 2021-12-16 03:00:00 ┆ 2021-12-16 02:00:00 ┆ [4] │ └────────┴─────────────────────┴─────────────────────┴─────────────────────┴───────────┘ Dynamic group by on an index column >>> df = pl.DataFrame( ... { ... "idx": pl.int_range(0, 6, eager=True), ... "A": ["A", "A", "B", "B", "B", "C"], ... } ... ) >>> ( ... df.group_by_dynamic( ... "idx", ... every="2i", ... period="3i", ... include_boundaries=True, ... closed="right", ... ).agg(pl.col("A").alias("A_agg_list")) ... ) shape: (4, 4) ┌─────────────────┬─────────────────┬─────┬─────────────────┐ │ _lower_boundary ┆ _upper_boundary ┆ idx ┆ A_agg_list │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 ┆ list[str] │ ╞═════════════════╪═════════════════╪═════╪═════════════════╡ │ -2 ┆ 1 ┆ -2 ┆ ["A", "A"] │ │ 0 ┆ 3 ┆ 0 ┆ ["A", "B", "B"] │ │ 2 ┆ 5 ┆ 2 ┆ ["B", "B", "C"] │ │ 4 ┆ 7 ┆ 4 ┆ ["C"] │ └─────────────────┴─────────────────┴─────┴─────────────────┘ ) reveryrrbrBrrrr)r;) rrrrrbrrrBrrs rgroup_by_dynamicDataFrame.group_by_dynamics0f  %1  r)rr=cUc/n[U[5(aU/n[U5nURURR X1X$55$)u Upsample a DataFrame at a regular frequency. The `every` argument is created with the following string language: - 1ns (1 nanosecond) - 1us (1 microsecond) - 1ms (1 millisecond) - 1s (1 second) - 1m (1 minute) - 1h (1 hour) - 1d (1 calendar day) - 1w (1 calendar week) - 1mo (1 calendar month) - 1q (1 calendar quarter) - 1y (1 calendar year) - 1i (1 index count) Or combine them: - "3d12h4m25s" # 3 days, 12 hours, 4 minutes, and 25 seconds By "calendar day", we mean the corresponding time on the next day (which may not be 24 hours, due to daylight savings). Similarly for "calendar week", "calendar month", "calendar quarter", and "calendar year". .. versionchanged:: 0.20.14 The `by` parameter was renamed `group_by`. Parameters ---------- time_column Time column will be used to determine a date_range. Note that this column has to be sorted for the output to make sense. every Interval will start 'every' duration. group_by First group by these columns and then upsample for every group. maintain_order Keep the ordering predictable. This is slower. Returns ------- DataFrame Result will be sorted by `time_column` (but note that if `group_by` columns are passed, it will only be sorted within each group). Examples -------- Upsample a DataFrame by a certain interval. >>> from datetime import datetime >>> df = pl.DataFrame( ... { ... "time": [ ... datetime(2021, 2, 1), ... datetime(2021, 4, 1), ... datetime(2021, 5, 1), ... datetime(2021, 6, 1), ... ], ... "groups": ["A", "B", "A", "B"], ... "values": [0, 1, 2, 3], ... } ... ).set_sorted("time") >>> df.upsample( ... time_column="time", every="1mo", group_by="groups", maintain_order=True ... ).select(pl.all().fill_null(strategy="forward")) shape: (7, 3) ┌─────────────────────┬────────┬────────┐ │ time ┆ groups ┆ values │ │ --- ┆ --- ┆ --- │ │ datetime[μs] ┆ str ┆ i64 │ ╞═════════════════════╪════════╪════════╡ │ 2021-02-01 00:00:00 ┆ A ┆ 0 │ │ 2021-03-01 00:00:00 ┆ A ┆ 0 │ │ 2021-04-01 00:00:00 ┆ A ┆ 0 │ │ 2021-05-01 00:00:00 ┆ A ┆ 2 │ │ 2021-04-01 00:00:00 ┆ B ┆ 1 │ │ 2021-05-01 00:00:00 ┆ B ┆ 1 │ │ 2021-06-01 00:00:00 ┆ B ┆ 3 │ └─────────────────────┴────────┴────────┘ )rrr#rrupsample)r time_columnrrr=s rrDataFrame.upsamplesSv  H h $ $ zH(/ HH  hU K  rbackward_rightleft_onright_ononby_leftby_rightr`strategyrM toleranceallow_parallelforce_parallelcoalesceallow_exact_matchescheck_sortednessrcN[X5 Ub@[U[[R45(dS[ U5<3n[ U5eO~[U[[R45(dS[ U5<3n[ U5e[U[[R45(dS[ U5<3n[ U5eSSKJn UR5RUR5UUUUUUUU U U U U UUS9RUR5S9$)u0 Perform an asof join. This is similar to a left-join except that we match on nearest key rather than equal keys. Both DataFrames must be sorted by the `on` key (within each `by` group, if specified). For each row in the left DataFrame: - A "backward" search selects the last row in the right DataFrame whose 'on' key is less than or equal to the left's key. - A "forward" search selects the first row in the right DataFrame whose 'on' key is greater than or equal to the left's key. - A "nearest" search selects the last row in the right DataFrame whose value is nearest to the left's key. String keys are not currently supported for a nearest search. The default is "backward". Parameters ---------- other Lazy DataFrame to join with. left_on Join column of the left DataFrame. right_on Join column of the right DataFrame. on Join column of both DataFrames. If set, `left_on` and `right_on` should be None. by Join on these columns before doing asof join by_left Join on these columns before doing asof join by_right Join on these columns before doing asof join strategy : {'backward', 'forward', 'nearest'} Join strategy. suffix Suffix to append to columns with a duplicate name. tolerance Numeric tolerance. By setting this the join will only be done if the near keys are within this distance. If an asof join is done on columns of dtype "Date", "Datetime", "Duration" or "Time", use either a datetime.timedelta object or the following string language: - 1ns (1 nanosecond) - 1us (1 microsecond) - 1ms (1 millisecond) - 1s (1 second) - 1m (1 minute) - 1h (1 hour) - 1d (1 calendar day) - 1w (1 calendar week) - 1mo (1 calendar month) - 1q (1 calendar quarter) - 1y (1 calendar year) Or combine them: "3d12h4m25s" # 3 days, 12 hours, 4 minutes, and 25 seconds By "calendar day", we mean the corresponding time on the next day (which may not be 24 hours, due to daylight savings). Similarly for "calendar week", "calendar month", "calendar quarter", and "calendar year". allow_parallel Allow the physical plan to optionally evaluate the computation of both DataFrames up to the join in parallel. force_parallel Force the physical plan to evaluate the computation of both DataFrames up to the join in parallel. coalesce Coalescing behavior (merging of `on` / `left_on` / `right_on` columns): - *True*: Always coalesce join columns. - *False*: Never coalesce join columns. Note that joining on any other expressions than `col` will turn off coalescing. allow_exact_matches Whether exact matches are valid join predicates. - If True, allow matching with the same ``on`` value (i.e. less-than-or-equal-to / greater-than-or-equal-to) - If False, don't match the same ``on`` value (i.e., strictly less-than / strictly greater-than). check_sortedness Check the sortedness of the asof keys. If the keys are not sorted Polars will error. Currently, sortedness cannot be checked if 'by' groups are provided. Examples -------- >>> from datetime import date >>> gdp = pl.DataFrame( ... { ... "date": pl.date_range( ... date(2016, 1, 1), ... date(2020, 1, 1), ... "1y", ... eager=True, ... ), ... "gdp": [4164, 4411, 4566, 4696, 4827], ... } ... ) >>> gdp shape: (5, 2) ┌────────────┬──────┐ │ date ┆ gdp │ │ --- ┆ --- │ │ date ┆ i64 │ ╞════════════╪══════╡ │ 2016-01-01 ┆ 4164 │ │ 2017-01-01 ┆ 4411 │ │ 2018-01-01 ┆ 4566 │ │ 2019-01-01 ┆ 4696 │ │ 2020-01-01 ┆ 4827 │ └────────────┴──────┘ >>> population = pl.DataFrame( ... { ... "date": [date(2016, 3, 1), date(2018, 8, 1), date(2019, 1, 1)], ... "population": [82.19, 82.66, 83.12], ... } ... ).sort("date") >>> population shape: (3, 2) ┌────────────┬────────────┐ │ date ┆ population │ │ --- ┆ --- │ │ date ┆ f64 │ ╞════════════╪════════════╡ │ 2016-03-01 ┆ 82.19 │ │ 2018-08-01 ┆ 82.66 │ │ 2019-01-01 ┆ 83.12 │ └────────────┴────────────┘ Note how the dates don't quite match. If we join them using `join_asof` and `strategy='backward'`, then each date from `population` which doesn't have an exact match is matched with the closest earlier date from `gdp`: >>> population.join_asof(gdp, on="date", strategy="backward") shape: (3, 3) ┌────────────┬────────────┬──────┐ │ date ┆ population ┆ gdp │ │ --- ┆ --- ┆ --- │ │ date ┆ f64 ┆ i64 │ ╞════════════╪════════════╪══════╡ │ 2016-03-01 ┆ 82.19 ┆ 4164 │ │ 2018-08-01 ┆ 82.66 ┆ 4566 │ │ 2019-01-01 ┆ 83.12 ┆ 4696 │ └────────────┴────────────┴──────┘ Note how: - date `2016-03-01` from `population` is matched with `2016-01-01` from `gdp`; - date `2018-08-01` from `population` is matched with `2018-01-01` from `gdp`. You can verify this by passing `coalesce=False`: >>> population.join_asof(gdp, on="date", strategy="backward", coalesce=False) shape: (3, 4) ┌────────────┬────────────┬────────────┬──────┐ │ date ┆ population ┆ date_right ┆ gdp │ │ --- ┆ --- ┆ --- ┆ --- │ │ date ┆ f64 ┆ date ┆ i64 │ ╞════════════╪════════════╪════════════╪══════╡ │ 2016-03-01 ┆ 82.19 ┆ 2016-01-01 ┆ 4164 │ │ 2018-08-01 ┆ 82.66 ┆ 2018-01-01 ┆ 4566 │ │ 2019-01-01 ┆ 83.12 ┆ 2019-01-01 ┆ 4696 │ └────────────┴────────────┴────────────┴──────┘ If we instead use `strategy='forward'`, then each date from `population` which doesn't have an exact match is matched with the closest later date from `gdp`: >>> population.join_asof(gdp, on="date", strategy="forward") shape: (3, 3) ┌────────────┬────────────┬──────┐ │ date ┆ population ┆ gdp │ │ --- ┆ --- ┆ --- │ │ date ┆ f64 ┆ i64 │ ╞════════════╪════════════╪══════╡ │ 2016-03-01 ┆ 82.19 ┆ 4411 │ │ 2018-08-01 ┆ 82.66 ┆ 4696 │ │ 2019-01-01 ┆ 83.12 ┆ 4696 │ └────────────┴────────────┴──────┘ Note how: - date `2016-03-01` from `population` is matched with `2017-01-01` from `gdp`; - date `2018-08-01` from `population` is matched with `2019-01-01` from `gdp`. Finally, `strategy='nearest'` gives us a mix of the two results above, as each date from `population` which doesn't have an exact match is matched with the closest date from `gdp`, regardless of whether it's earlier or later: >>> population.join_asof(gdp, on="date", strategy="nearest") shape: (3, 3) ┌────────────┬────────────┬──────┐ │ date ┆ population ┆ gdp │ │ --- ┆ --- ┆ --- │ │ date ┆ f64 ┆ i64 │ ╞════════════╪════════════╪══════╡ │ 2016-03-01 ┆ 82.19 ┆ 4164 │ │ 2018-08-01 ┆ 82.66 ┆ 4696 │ │ 2019-01-01 ┆ 83.12 ┆ 4696 │ └────────────┴────────────┴──────┘ Note how: - date `2016-03-01` from `population` is matched with `2016-01-01` from `gdp`; - date `2018-08-01` from `population` is matched with `2019-01-01` from `gdp`. They `by` argument allows joining on another column first, before the asof join. In this example we join by `country` first, then asof join by date, as above. >>> gdp_dates = pl.date_range( # fmt: skip ... date(2016, 1, 1), date(2020, 1, 1), "1y", eager=True ... ) >>> gdp2 = pl.DataFrame( ... { ... "country": ["Germany"] * 5 + ["Netherlands"] * 5, ... "date": pl.concat([gdp_dates, gdp_dates]), ... "gdp": [4164, 4411, 4566, 4696, 4827, 784, 833, 914, 910, 909], ... } ... ).sort("country", "date") >>> >>> gdp2 shape: (10, 3) ┌─────────────┬────────────┬──────┐ │ country ┆ date ┆ gdp │ │ --- ┆ --- ┆ --- │ │ str ┆ date ┆ i64 │ ╞═════════════╪════════════╪══════╡ │ Germany ┆ 2016-01-01 ┆ 4164 │ │ Germany ┆ 2017-01-01 ┆ 4411 │ │ Germany ┆ 2018-01-01 ┆ 4566 │ │ Germany ┆ 2019-01-01 ┆ 4696 │ │ Germany ┆ 2020-01-01 ┆ 4827 │ │ Netherlands ┆ 2016-01-01 ┆ 784 │ │ Netherlands ┆ 2017-01-01 ┆ 833 │ │ Netherlands ┆ 2018-01-01 ┆ 914 │ │ Netherlands ┆ 2019-01-01 ┆ 910 │ │ Netherlands ┆ 2020-01-01 ┆ 909 │ └─────────────┴────────────┴──────┘ >>> pop2 = pl.DataFrame( ... { ... "country": ["Germany"] * 3 + ["Netherlands"] * 3, ... "date": [ ... date(2016, 3, 1), ... date(2018, 8, 1), ... date(2019, 1, 1), ... date(2016, 3, 1), ... date(2018, 8, 1), ... date(2019, 1, 1), ... ], ... "population": [82.19, 82.66, 83.12, 17.11, 17.32, 17.40], ... } ... ).sort("country", "date") >>> >>> pop2 shape: (6, 3) ┌─────────────┬────────────┬────────────┐ │ country ┆ date ┆ population │ │ --- ┆ --- ┆ --- │ │ str ┆ date ┆ f64 │ ╞═════════════╪════════════╪════════════╡ │ Germany ┆ 2016-03-01 ┆ 82.19 │ │ Germany ┆ 2018-08-01 ┆ 82.66 │ │ Germany ┆ 2019-01-01 ┆ 83.12 │ │ Netherlands ┆ 2016-03-01 ┆ 17.11 │ │ Netherlands ┆ 2018-08-01 ┆ 17.32 │ │ Netherlands ┆ 2019-01-01 ┆ 17.4 │ └─────────────┴────────────┴────────────┘ >>> pop2.join_asof(gdp2, by="country", on="date", strategy="nearest") shape: (6, 4) ┌─────────────┬────────────┬────────────┬──────┐ │ country ┆ date ┆ population ┆ gdp │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ date ┆ f64 ┆ i64 │ ╞═════════════╪════════════╪════════════╪══════╡ │ Germany ┆ 2016-03-01 ┆ 82.19 ┆ 4164 │ │ Germany ┆ 2018-08-01 ┆ 82.66 ┆ 4696 │ │ Germany ┆ 2019-01-01 ┆ 83.12 ┆ 4696 │ │ Netherlands ┆ 2016-03-01 ┆ 17.11 ┆ 784 │ │ Netherlands ┆ 2018-08-01 ┆ 17.32 ┆ 910 │ │ Netherlands ┆ 2019-01-01 ┆ 17.4 ┆ 910 │ └─────────────┴────────────┴────────────┴──────┘ z%expected `on` to be str or Expr, got z*expected `left_on` to be str or Expr, got z+expected `right_on` to be str or Expr, got rrrr) r4rrrrsr3rrrr join_asofrr)rr:rrrrrr`rrMrrrrrrrrs rrDataFrame.join_asofCs#r $& >b3.11;>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6.0, 7.0, 8.0], ... "ham": ["a", "b", "c"], ... } ... ) >>> other_df = pl.DataFrame( ... { ... "apple": ["x", "y", "z"], ... "ham": ["a", "b", "d"], ... } ... ) >>> df.join(other_df, on="ham") shape: (2, 4) ┌─────┬─────┬─────┬───────┐ │ foo ┆ bar ┆ ham ┆ apple │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str ┆ str │ ╞═════╪═════╪═════╪═══════╡ │ 1 ┆ 6.0 ┆ a ┆ x │ │ 2 ┆ 7.0 ┆ b ┆ y │ └─────┴─────┴─────┴───────┘ >>> df.join(other_df, on="ham", how="full") shape: (4, 5) ┌──────┬──────┬──────┬───────┬───────────┐ │ foo ┆ bar ┆ ham ┆ apple ┆ ham_right │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str ┆ str ┆ str │ ╞══════╪══════╪══════╪═══════╪═══════════╡ │ 1 ┆ 6.0 ┆ a ┆ x ┆ a │ │ 2 ┆ 7.0 ┆ b ┆ y ┆ b │ │ null ┆ null ┆ null ┆ z ┆ d │ │ 3 ┆ 8.0 ┆ c ┆ null ┆ null │ └──────┴──────┴──────┴───────┴───────────┘ >>> df.join(other_df, on="ham", how="full", coalesce=True) shape: (4, 4) ┌──────┬──────┬─────┬───────┐ │ foo ┆ bar ┆ ham ┆ apple │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str ┆ str │ ╞══════╪══════╪═════╪═══════╡ │ 1 ┆ 6.0 ┆ a ┆ x │ │ 2 ┆ 7.0 ┆ b ┆ y │ │ null ┆ null ┆ d ┆ z │ │ 3 ┆ 8.0 ┆ c ┆ null │ └──────┴──────┴─────┴───────┘ >>> df.join(other_df, on="ham", how="left") shape: (3, 4) ┌─────┬─────┬─────┬───────┐ │ foo ┆ bar ┆ ham ┆ apple │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str ┆ str │ ╞═════╪═════╪═════╪═══════╡ │ 1 ┆ 6.0 ┆ a ┆ x │ │ 2 ┆ 7.0 ┆ b ┆ y │ │ 3 ┆ 8.0 ┆ c ┆ null │ └─────┴─────┴─────┴───────┘ >>> df.join(other_df, on="ham", how="semi") shape: (2, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6.0 ┆ a │ │ 2 ┆ 7.0 ┆ b │ └─────┴─────┴─────┘ >>> df.join(other_df, on="ham", how="anti") shape: (1, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str │ ╞═════╪═════╪═════╡ │ 3 ┆ 8.0 ┆ c │ └─────┴─────┴─────┘ >>> df.join(other_df, how="cross") shape: (9, 5) ┌─────┬─────┬─────┬───────┬───────────┐ │ foo ┆ bar ┆ ham ┆ apple ┆ ham_right │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str ┆ str ┆ str │ ╞═════╪═════╪═════╪═══════╪═══════════╡ │ 1 ┆ 6.0 ┆ a ┆ x ┆ a │ │ 1 ┆ 6.0 ┆ a ┆ y ┆ b │ │ 1 ┆ 6.0 ┆ a ┆ z ┆ d │ │ 2 ┆ 7.0 ┆ b ┆ x ┆ a │ │ 2 ┆ 7.0 ┆ b ┆ y ┆ b │ │ 2 ┆ 7.0 ┆ b ┆ z ┆ d │ │ 3 ┆ 8.0 ┆ c ┆ x ┆ a │ │ 3 ┆ 8.0 ┆ c ┆ y ┆ b │ │ 3 ┆ 8.0 ┆ c ┆ z ┆ d │ └─────┴─────┴─────┴───────┴───────────┘ Notes ----- For joining on columns with categorical data, see :class:`polars.StringCache`. rr) r:rrrrBrMrrrr=r)r4rrrrrr) rr:rrBrrrMrrrr=rs rrDataFrame.joinsjD $&< IIK Tjjl!!'!- W=#7#7#9W : r)rMc[X5 SSKJn UR5R"UR5/UQ7SU06R UR 5S9$)u Perform a join based on one or multiple (in)equality predicates. This performs an inner join, so only rows where all predicates are true are included in the result, and a row from either DataFrame may be included multiple times in the result. .. note:: The row order of the input DataFrames is not preserved. .. warning:: This functionality is experimental. It may be changed at any point without it being considered a breaking change. Parameters ---------- other DataFrame to join with. *predicates (In)Equality condition to join the two tables on. When a column name occurs in both tables, the proper suffix must be applied in the predicate. suffix Suffix to append to columns with a duplicate name. Examples -------- Join two dataframes together based on two predicates which get AND-ed together. >>> east = pl.DataFrame( ... { ... "id": [100, 101, 102], ... "dur": [120, 140, 160], ... "rev": [12, 14, 16], ... "cores": [2, 8, 4], ... } ... ) >>> west = pl.DataFrame( ... { ... "t_id": [404, 498, 676, 742], ... "time": [90, 130, 150, 170], ... "cost": [9, 13, 15, 16], ... "cores": [4, 2, 1, 4], ... } ... ) >>> east.join_where( ... west, ... pl.col("dur") < pl.col("time"), ... pl.col("rev") < pl.col("cost"), ... ) shape: (5, 8) ┌─────┬─────┬─────┬───────┬──────┬──────┬──────┬─────────────┐ │ id ┆ dur ┆ rev ┆ cores ┆ t_id ┆ time ┆ cost ┆ cores_right │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╪═══════╪══════╪══════╪══════╪═════════════╡ │ 100 ┆ 120 ┆ 12 ┆ 2 ┆ 498 ┆ 130 ┆ 13 ┆ 2 │ │ 100 ┆ 120 ┆ 12 ┆ 2 ┆ 676 ┆ 150 ┆ 15 ┆ 1 │ │ 100 ┆ 120 ┆ 12 ┆ 2 ┆ 742 ┆ 170 ┆ 16 ┆ 4 │ │ 101 ┆ 140 ┆ 14 ┆ 8 ┆ 676 ┆ 150 ┆ 15 ┆ 1 │ │ 101 ┆ 140 ┆ 14 ┆ 8 ┆ 742 ┆ 170 ┆ 16 ┆ 4 │ └─────┴─────┴─────┴───────┴──────┴──────┴──────┴─────────────┘ To OR them together, use a single expression and the `|` operator. >>> east.join_where( ... west, ... (pl.col("dur") < pl.col("time")) | (pl.col("rev") < pl.col("cost")), ... ) shape: (6, 8) ┌─────┬─────┬─────┬───────┬──────┬──────┬──────┬─────────────┐ │ id ┆ dur ┆ rev ┆ cores ┆ t_id ┆ time ┆ cost ┆ cores_right │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╪═══════╪══════╪══════╪══════╪═════════════╡ │ 100 ┆ 120 ┆ 12 ┆ 2 ┆ 498 ┆ 130 ┆ 13 ┆ 2 │ │ 100 ┆ 120 ┆ 12 ┆ 2 ┆ 676 ┆ 150 ┆ 15 ┆ 1 │ │ 100 ┆ 120 ┆ 12 ┆ 2 ┆ 742 ┆ 170 ┆ 16 ┆ 4 │ │ 101 ┆ 140 ┆ 14 ┆ 8 ┆ 676 ┆ 150 ┆ 15 ┆ 1 │ │ 101 ┆ 140 ┆ 14 ┆ 8 ┆ 742 ┆ 170 ┆ 16 ┆ 4 │ │ 102 ┆ 160 ┆ 16 ┆ 4 ┆ 742 ┆ 170 ┆ 16 ┆ 4 │ └─────┴─────┴─────┴───────┴──────┴──────┴──────┴─────────────┘ rrrMr)r4rrr join_whererr)rr:rMrrs rrDataFrame.join_wheresct $&< IIK Z       W=#7#7#9W : r)inference_sizecURRXU5upEU(aURU5$[U5R 5$)u& Apply a custom/user-defined function (UDF) over the rows of the DataFrame. .. warning:: This method is much slower than the native expressions API. Only use it if you cannot implement your logic otherwise. The UDF will receive each row as a tuple of values: `udf(row)`. Implementing logic using a Python function is almost always *significantly* slower and more memory intensive than implementing the same logic using the native expression API because: - The native expression engine runs in Rust; UDFs run in Python. - Use of Python UDFs forces the DataFrame to be materialized in memory. - Polars-native expressions can be parallelised (UDFs typically cannot). - Polars-native expressions can be logically optimised (UDFs cannot). Wherever possible you should strongly prefer the native expression API to achieve the best performance. Parameters ---------- function Custom function or lambda. return_dtype Output type of the operation. If none given, Polars tries to infer the type. inference_size Only used in the case when the custom function returns rows. This uses the first `n` rows to determine the output schema. Notes ----- * The frame-level `map_rows` cannot track column names (as the UDF is a black-box that may arbitrarily drop, rearrange, transform, or add new columns); if you want to apply a UDF such that column names are preserved, you should use the expression-level `map_elements` syntax instead. * If your function is expensive and you don't want it to be called more than once for a given input, consider applying an `@lru_cache` decorator to it. If your data is suitable you may achieve *significant* speedups. Examples -------- >>> df = pl.DataFrame({"foo": [1, 2, 3], "bar": [-1, 5, 8]}) Return a DataFrame by mapping each row to a tuple: >>> df.map_rows(lambda t: (t[0] * 2, t[1] * 3)) shape: (3, 2) ┌──────────┬──────────┐ │ column_0 ┆ column_1 │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞══════════╪══════════╡ │ 2 ┆ -3 │ │ 4 ┆ 15 │ │ 6 ┆ 24 │ └──────────┴──────────┘ However, it is much better to implement this with a native expression: >>> df.select( ... pl.col("foo") * 2, ... pl.col("bar") * 3, ... ) # doctest: +IGNORE_RESULT Return a DataFrame with a single column by mapping each row to a scalar: >>> df.map_rows(lambda t: (t[0] * 2 + t[1])) shape: (3, 1) ┌─────┐ │ map │ │ --- │ │ i64 │ ╞═════╡ │ 1 │ │ 9 │ │ 14 │ └─────┘ In this case it is better to use the following native expression: >>> df.select(pl.col("foo") * 2 + pl.col("bar")) # doctest: +IGNORE_RESULT )rmap_rowsrr9to_frame)rrx return_dtyperr=is_dfs rrDataFrame.map_rows sB@XX&&x~N  ??3' '#;'') )r)in_placercd[U[5(dUR5nU(a7URR UVs/sHo3R PM sn5 U$UR URRUVs/sHo3R PM sn55$s snfs snf)uh Return a new DataFrame grown horizontally by stacking multiple Series to it. Parameters ---------- columns Series to stack. in_place Modify in place. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... } ... ) >>> x = pl.Series("apple", [10, 20, 30]) >>> df.hstack([x]) shape: (3, 4) ┌─────┬─────┬─────┬───────┐ │ foo ┆ bar ┆ ham ┆ apple │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str ┆ i64 │ ╞═════╪═════╪═════╪═══════╡ │ 1 ┆ 6 ┆ a ┆ 10 │ │ 2 ┆ 7 ┆ b ┆ 20 │ │ 3 ┆ 8 ┆ c ┆ 30 │ └─────┴─────┴─────┴───────┘ )rrrr hstack_mutrrhstack)rrrrds rrDataFrame.hstacki sF'4(())+G  HH  w 7w!w 7 8K??488??'3J'QDD'3J#KL L!84Ks B(B- c[X5 U(a'URRUR5 U$URURR UR55$)ua Grow this DataFrame vertically by stacking a DataFrame to it. Parameters ---------- other DataFrame to stack. in_place Modify in place. See Also -------- extend Examples -------- >>> df1 = pl.DataFrame( ... { ... "foo": [1, 2], ... "bar": [6, 7], ... "ham": ["a", "b"], ... } ... ) >>> df2 = pl.DataFrame( ... { ... "foo": [3, 4], ... "bar": [8, 9], ... "ham": ["c", "d"], ... } ... ) >>> df1.vstack(df2) shape: (4, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ │ 2 ┆ 7 ┆ b │ │ 3 ┆ 8 ┆ c │ │ 4 ┆ 9 ┆ d │ └─────┴─────┴─────┘ )r4r vstack_mutrvstack)rr:rs rrDataFrame.vstack sKX $&  HH   *Ktxxuyy9::rcf[X5 URRUR5 U$)u Extend the memory backed by this `DataFrame` with the values from `other`. Different from `vstack` which adds the chunks from `other` to the chunks of this `DataFrame`, `extend` appends the data from `other` to the underlying memory locations and thus may cause a reallocation. If this does not cause a reallocation, the resulting data structure will not have any extra chunks and thus will yield faster queries. Prefer `extend` over `vstack` when you want to do a query after a single append. For instance, during online operations where you add `n` rows and rerun a query. Prefer `vstack` over `extend` when you want to append many times before doing a query. For instance, when you read in multiple files and want to store them in a single `DataFrame`. In the latter case, finish the sequence of `vstack` operations with a `rechunk`. Parameters ---------- other DataFrame to vertically add. Warnings -------- This method modifies the dataframe in-place. The dataframe is returned for convenience only. See Also -------- vstack Examples -------- >>> df1 = pl.DataFrame({"foo": [1, 2, 3], "bar": [4, 5, 6]}) >>> df2 = pl.DataFrame({"foo": [10, 20, 30], "bar": [40, 50, 60]}) >>> df1.extend(df2) shape: (6, 2) ┌─────┬─────┐ │ foo ┆ bar │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 4 │ │ 2 ┆ 5 │ │ 3 ┆ 6 │ │ 10 ┆ 40 │ │ 20 ┆ 50 │ │ 30 ┆ 60 │ └─────┴─────┘ )r4rextendrss rrDataFrame.extend s&j $&  " rcSSKJn UR5R"USU06R UR 5S9$)u Remove columns from the dataframe. Parameters ---------- *columns Names of the columns that should be removed from the dataframe. Accepts column selector input. strict Validate that all column names exist in the current schema, and throw an exception if any do not. Examples -------- Drop a single column by passing the name of that column. >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6.0, 7.0, 8.0], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.drop("ham") shape: (3, 2) ┌─────┬─────┐ │ foo ┆ bar │ │ --- ┆ --- │ │ i64 ┆ f64 │ ╞═════╪═════╡ │ 1 ┆ 6.0 │ │ 2 ┆ 7.0 │ │ 3 ┆ 8.0 │ └─────┴─────┘ Drop multiple columns by passing a list of column names. >>> df.drop(["bar", "ham"]) shape: (3, 1) ┌─────┐ │ foo │ │ --- │ │ i64 │ ╞═════╡ │ 1 │ │ 2 │ │ 3 │ └─────┘ Drop multiple columns by passing a selector. >>> import polars.selectors as cs >>> df.drop(cs.numeric()) shape: (3, 1) ┌─────┐ │ ham │ │ --- │ │ str │ ╞═════╡ │ a │ │ b │ │ c │ └─────┘ Use positional arguments to drop multiple columns. >>> df.drop("foo", "ham") shape: (3, 1) ┌─────┐ │ bar │ │ --- │ │ f64 │ ╞═════╡ │ 6.0 │ │ 7.0 │ │ 8.0 │ └─────┘ rrrr)rrrrbrr)rrrrs rrbDataFrame.drop!sGf = IIK T  +#) + W=#7#7#9W : rcJ[URRU55$)a Drop a single column in-place and return the dropped column. Parameters ---------- name Name of the column to drop. Returns ------- Series The dropped column. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.drop_in_place("ham") shape: (3,) Series: 'ham' [str] [ "a" "b" "c" ] )r9r drop_in_placers rrDataFrame.drop_in_place[!s@dhh,,T233rc~SSKJn UR5RXS9R UR 5S9$)uc Cast DataFrame column(s) to the specified dtype(s). Parameters ---------- dtypes Mapping of column names (or selector) to dtypes, or a single dtype to which all columns will be cast. strict Raise if cast is invalid on rows after predicates are pushed down. If `False`, invalid casts will produce null values. Examples -------- >>> from datetime import date >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6.0, 7.0, 8.0], ... "ham": [date(2020, 1, 2), date(2021, 3, 4), date(2022, 5, 6)], ... } ... ) Cast specific frame columns to the specified dtypes: >>> df.cast({"foo": pl.Float32, "bar": pl.UInt8}) shape: (3, 3) ┌─────┬─────┬────────────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ f32 ┆ u8 ┆ date │ ╞═════╪═════╪════════════╡ │ 1.0 ┆ 6 ┆ 2020-01-02 │ │ 2.0 ┆ 7 ┆ 2021-03-04 │ │ 3.0 ┆ 8 ┆ 2022-05-06 │ └─────┴─────┴────────────┘ Cast all frame columns matching one dtype (or dtype group) to another dtype: >>> df.cast({pl.Date: pl.Datetime}) shape: (3, 3) ┌─────┬─────┬─────────────────────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ datetime[μs] │ ╞═════╪═════╪═════════════════════╡ │ 1 ┆ 6.0 ┆ 2020-01-02 00:00:00 │ │ 2 ┆ 7.0 ┆ 2021-03-04 00:00:00 │ │ 3 ┆ 8.0 ┆ 2022-05-06 00:00:00 │ └─────┴─────┴─────────────────────┘ Use selectors to define the columns being cast: >>> import polars.selectors as cs >>> df.cast({cs.numeric(): pl.UInt32, cs.temporal(): pl.String}) shape: (3, 3) ┌─────┬─────┬────────────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ u32 ┆ u32 ┆ str │ ╞═════╪═════╪════════════╡ │ 1 ┆ 6 ┆ 2020-01-02 │ │ 2 ┆ 7 ┆ 2021-03-04 │ │ 3 ┆ 8 ┆ 2022-05-06 │ └─────┴─────┴────────────┘ Cast all frame columns to the specified dtype: >>> df.cast(pl.String).to_dict(as_series=False) {'foo': ['1', '2', '3'], 'bar': ['6.0', '7.0', '8.0'], 'ham': ['2020-01-02', '2021-03-04', '2022-05-06']} rrrr)rrrrrr)rrrrs rrDataFrame.cast}!s9h = IIK T&T ( W=#7#7#9W : rc ZUS:aSU3n[U5eUS:Xa)URURR55$UR UR R 5VVs0sH*up4U[R"X4S9RSU5_M, snn5$s snnf)u Create an empty (n=0) or `n`-row null-filled (n>0) copy of the DataFrame. Returns a `n`-row null-filled DataFrame with an identical schema. `n` can be greater than the current number of rows in the DataFrame. Parameters ---------- n Number of (null-filled) rows to return in the cleared frame. See Also -------- clone : Cheap deepcopy/clone. Examples -------- >>> df = pl.DataFrame( ... { ... "a": [None, 2, 3, 4], ... "b": [0.5, None, 2.5, 13], ... "c": [True, True, False, None], ... } ... ) >>> df.clear() shape: (0, 3) ┌─────┬─────┬──────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ bool │ ╞═════╪═════╪══════╡ └─────┴─────┴──────┘ >>> df.clear(n=2) shape: (2, 3) ┌──────┬──────┬──────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ bool │ ╞══════╪══════╪══════╡ │ null ┆ null ┆ null │ │ null ┆ null ┆ null │ └──────┴──────┴──────┘ rz.`n` should be greater than or equal to 0, got )rr+N) rrrclear __class__rrarruextend_constant)rrQrnmrgs rrDataFrame.clear!sZ q5B1#FCS/ ! 6??488>>#34 4~~#kk//1 1FBBII20@@qII1    s.1B' cTURURR55$)u8 Create a copy of this DataFrame. This is a cheap operation that does not copy data. See Also -------- clear : Create an empty copy of the current DataFrame, with identical schema but no data. Examples -------- >>> df = pl.DataFrame( ... { ... "a": [1, 2, 3, 4], ... "b": [0.5, 4, 10, 13], ... "c": [True, True, False, True], ... } ... ) >>> df.clone() shape: (4, 3) ┌─────┬──────┬───────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ bool │ ╞═════╪══════╪═══════╡ │ 1 ┆ 0.5 ┆ true │ │ 2 ┆ 4.0 ┆ true │ │ 3 ┆ 10.0 ┆ false │ │ 4 ┆ 13.0 ┆ true │ └─────┴──────┴───────┘ )rrrr s rrDataFrame.clone"sBtxx~~/00rctURR5Vs/sHn[U5PM sn$s snf)a Get the DataFrame as a List of Series. Examples -------- >>> df = pl.DataFrame({"foo": [1, 2, 3], "bar": [4, 5, 6]}) >>> df.get_columns() [shape: (3,) Series: 'foo' [i64] [ 1 2 3 ], shape: (3,) Series: 'bar' [i64] [ 4 5 6 ]] >>> df = pl.DataFrame( ... { ... "a": [1, 2, 3, 4], ... "b": [0.5, 4, 10, 13], ... "c": [True, True, False, True], ... } ... ) >>> df.get_columns() [shape: (4,) Series: 'a' [i64] [ 1 2 3 4 ], shape: (4,) Series: 'b' [f64] [ 0.5 4.0 10.0 13.0 ], shape: (4,) Series: 'c' [bool] [ true true false true ]] rrr9rrds rrDataFrame.get_columns6"s0j$(88#7#7#9:#9aq #9:::s5rcgrrrrrs rrDataFrame.get_columnm"sUXrcgrrrs rrrp"rrc[URRU55$![a U[LaeUs$f=f)aY Get a single column by name. Parameters ---------- name String name of the column to retrieve. default Value to return if the column does not exist; if not explicitly set and the column is not present a `ColumnNotFoundError` exception is raised. Returns ------- Series (or arbitrary default value, if specified). See Also -------- to_series Examples -------- >>> df = pl.DataFrame({"foo": [1, 2, 3], "bar": [4, 5, 6]}) >>> df.get_column("foo") shape: (3,) Series: 'foo' [i64] [ 1 2 3 ] Missing column handling; can optionally provide an arbitrary default value to the method (otherwise a `ColumnNotFoundError` exception is raised). >>> df.get_column("baz", default=pl.Series("baz", ["?", "?", "?"])) shape: (3,) Series: 'baz' [str] [ "?" "?" "?" ] >>> res = df.get_column("baz", default=None) >>> res is None True )r9rrr\r0rs rrrs"sAb $((--d34 4" *$N s #&??matches_supertypecSSKJn UR5RXX4S9R UR 5S9$)u  Fill null values using the specified value or strategy. Parameters ---------- value Value used to fill null values. strategy : {None, 'forward', 'backward', 'min', 'max', 'mean', 'zero', 'one'} Strategy used to fill null values. limit Number of consecutive null values to fill when using the 'forward' or 'backward' strategy. matches_supertype Fill all matching supertype of the fill `value`. Returns ------- DataFrame DataFrame with None values replaced by the filling strategy. See Also -------- fill_nan Notes ----- A null value is not the same as a NaN value. To fill NaN values, use :func:`fill_nan`. Examples -------- >>> df = pl.DataFrame( ... { ... "a": [1, 2, None, 4], ... "b": [0.5, 4, None, 13], ... } ... ) >>> df.fill_null(99) shape: (4, 2) ┌─────┬──────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ f64 │ ╞═════╪══════╡ │ 1 ┆ 0.5 │ │ 2 ┆ 4.0 │ │ 99 ┆ 99.0 │ │ 4 ┆ 13.0 │ └─────┴──────┘ >>> df.fill_null(strategy="forward") shape: (4, 2) ┌─────┬──────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ f64 │ ╞═════╪══════╡ │ 1 ┆ 0.5 │ │ 2 ┆ 4.0 │ │ 2 ┆ 4.0 │ │ 4 ┆ 13.0 │ └─────┴──────┘ >>> df.fill_null(strategy="max") shape: (4, 2) ┌─────┬──────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ f64 │ ╞═════╪══════╡ │ 1 ┆ 0.5 │ │ 2 ┆ 4.0 │ │ 4 ┆ 13.0 │ │ 4 ┆ 13.0 │ └─────┴──────┘ >>> df.fill_null(strategy="zero") shape: (4, 2) ┌─────┬──────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ f64 │ ╞═════╪══════╡ │ 1 ┆ 0.5 │ │ 2 ┆ 4.0 │ │ 0 ┆ 0.0 │ │ 4 ┆ 13.0 │ └─────┴──────┘ rrrr)rrr fill_nullrr)rrrrmrrs rrDataFrame.fill_null"s;@ = IIK YuY S W=#7#7#9W : rcSSKJn UR5RU5R UR 5S9$)u9 Fill floating point NaN values by an Expression evaluation. Parameters ---------- value Value used to fill NaN values. Returns ------- DataFrame DataFrame with NaN values replaced by the given value. See Also -------- fill_null Notes ----- A NaN value is not the same as a null value. To fill null values, use :func:`fill_null`. Examples -------- >>> df = pl.DataFrame( ... { ... "a": [1.5, 2, float("nan"), 4], ... "b": [0.5, 4, float("nan"), 13], ... } ... ) >>> df.fill_nan(99) shape: (4, 2) ┌──────┬──────┐ │ a ┆ b │ │ --- ┆ --- │ │ f64 ┆ f64 │ ╞══════╪══════╡ │ 1.5 ┆ 0.5 │ │ 2.0 ┆ 4.0 │ │ 99.0 ┆ 99.0 │ │ 4.0 ┆ 13.0 │ └──────┴──────┘ rrr)rrrfill_nanrr)rrrs rrDataFrame.fill_nan#s6X =yy{##E*22AUAUAW2XXrcSSKJn UR5R"U/UQ76R UR 5S9$)u Explode the dataframe to long format by exploding the given columns. Parameters ---------- columns Column names, expressions, or a selector defining them. The underlying columns being exploded must be of the `List` or `Array` data type. *more_columns Additional names of columns to explode, specified as positional arguments. Returns ------- DataFrame Examples -------- >>> df = pl.DataFrame( ... { ... "letters": ["a", "a", "b", "c"], ... "numbers": [[1], [2, 3], [4, 5], [6, 7, 8]], ... } ... ) >>> df shape: (4, 2) ┌─────────┬───────────┐ │ letters ┆ numbers │ │ --- ┆ --- │ │ str ┆ list[i64] │ ╞═════════╪═══════════╡ │ a ┆ [1] │ │ a ┆ [2, 3] │ │ b ┆ [4, 5] │ │ c ┆ [6, 7, 8] │ └─────────┴───────────┘ >>> df.explode("numbers") shape: (8, 2) ┌─────────┬─────────┐ │ letters ┆ numbers │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═════════╪═════════╡ │ a ┆ 1 │ │ a ┆ 2 │ │ a ┆ 3 │ │ b ┆ 4 │ │ b ┆ 5 │ │ c ┆ 6 │ │ c ┆ 7 │ │ c ┆ 8 │ └─────────┴─────────┘ rrr)rrrexploderrrr more_columnsrs rrDataFrame.explodeC#sGr = IIK W  -+ - W=#7#7#9W : rr)rraggregate_functionr= sort_columnsrc [X5nUb [X5nUb [X5n[U[5(GaUS:Xa/[R"5R 5R nGOUS:Xa/[R"5R5R nGOUS:Xa/[R"5R5R nGOJUS:Xa/[R"5R5R nGOUS:Xa.[R"5R5R nOUS:Xa.[R"5R5R nOUS:Xa.[R"5R5R nOyUS :Xa [R"5R nOSUS :Xa*[S S S 9 [R"5R nO#SU<3n [U 5eUcSnO UR nUR!UR"R%UUUUUUU55$)u Create a spreadsheet-style pivot table as a DataFrame. Only available in eager mode. See "Examples" section below for how to do a "lazy pivot" if you know the unique column values in advance. .. versionchanged:: 1.0.0 The `columns` parameter was renamed `on`. Parameters ---------- on The column(s) whose values will be used as the new columns of the output DataFrame. index The column(s) that remain from the input to the output. The output DataFrame will have one row for each unique combination of the `index`'s values. If None, all remaining columns not specified on `on` and `values` will be used. At least one of `index` and `values` must be specified. values The existing column(s) of values which will be moved under the new columns from index. If an aggregation is specified, these are the values on which the aggregation will be computed. If None, all remaining columns not specified on `on` and `index` will be used. At least one of `index` and `values` must be specified. aggregate_function Choose from: - None: no aggregation takes place, will raise error if multiple values are in group. - A predefined aggregate function string, one of {'min', 'max', 'first', 'last', 'sum', 'mean', 'median', 'len'} - An expression to do the aggregation. The expression can only access data from the respective 'values' columns as generated by pivot, through `pl.element()`. maintain_order Ensure the values of `index` are sorted by discovery order. sort_columns Sort the transposed columns by name. Default is by order of discovery. separator Used as separator/delimiter in generated column names in case of multiple `values` columns. Returns ------- DataFrame Notes ----- In some other frameworks, you might know this operation as `pivot_wider`. Examples -------- You can use `pivot` to reshape a dataframe from "long" to "wide" format. For example, suppose we have a dataframe of test scores achieved by some students, where each row represents a distinct test. >>> df = pl.DataFrame( ... { ... "name": ["Cady", "Cady", "Karen", "Karen"], ... "subject": ["maths", "physics", "maths", "physics"], ... "test_1": [98, 99, 61, 58], ... "test_2": [100, 100, 60, 60], ... } ... ) >>> df shape: (4, 4) ┌───────┬─────────┬────────┬────────┐ │ name ┆ subject ┆ test_1 ┆ test_2 │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ str ┆ i64 ┆ i64 │ ╞═══════╪═════════╪════════╪════════╡ │ Cady ┆ maths ┆ 98 ┆ 100 │ │ Cady ┆ physics ┆ 99 ┆ 100 │ │ Karen ┆ maths ┆ 61 ┆ 60 │ │ Karen ┆ physics ┆ 58 ┆ 60 │ └───────┴─────────┴────────┴────────┘ Using `pivot`, we can reshape so we have one row per student, with different subjects as columns, and their `test_1` scores as values: >>> df.pivot("subject", index="name", values="test_1") shape: (2, 3) ┌───────┬───────┬─────────┐ │ name ┆ maths ┆ physics │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═══════╪═══════╪═════════╡ │ Cady ┆ 98 ┆ 99 │ │ Karen ┆ 61 ┆ 58 │ └───────┴───────┴─────────┘ You can use selectors too - here we include all test scores in the pivoted table: >>> import polars.selectors as cs >>> df.pivot("subject", values=cs.starts_with("test")) shape: (2, 5) ┌───────┬──────────────┬────────────────┬──────────────┬────────────────┐ │ name ┆ test_1_maths ┆ test_1_physics ┆ test_2_maths ┆ test_2_physics │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 ┆ i64 ┆ i64 │ ╞═══════╪══════════════╪════════════════╪══════════════╪════════════════╡ │ Cady ┆ 98 ┆ 99 ┆ 100 ┆ 100 │ │ Karen ┆ 61 ┆ 58 ┆ 60 ┆ 60 │ └───────┴──────────────┴────────────────┴──────────────┴────────────────┘ If you end up with multiple values per cell, you can specify how to aggregate them with `aggregate_function`: >>> df = pl.DataFrame( ... { ... "ix": [1, 1, 2, 2, 1, 2], ... "col": ["a", "a", "a", "a", "b", "b"], ... "foo": [0, 1, 2, 2, 7, 1], ... "bar": [0, 2, 0, 0, 9, 4], ... } ... ) >>> df.pivot("col", index="ix", aggregate_function="sum") shape: (2, 5) ┌─────┬───────┬───────┬───────┬───────┐ │ ix ┆ foo_a ┆ foo_b ┆ bar_a ┆ bar_b │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 │ ╞═════╪═══════╪═══════╪═══════╪═══════╡ │ 1 ┆ 1 ┆ 7 ┆ 2 ┆ 9 │ │ 2 ┆ 4 ┆ 1 ┆ 0 ┆ 4 │ └─────┴───────┴───────┴───────┴───────┘ You can also pass a custom aggregation function using :meth:`polars.element`: >>> df = pl.DataFrame( ... { ... "col1": ["a", "a", "a", "b", "b", "b"], ... "col2": ["x", "x", "x", "x", "y", "y"], ... "col3": [6, 7, 3, 2, 5, 7], ... } ... ) >>> df.pivot( ... "col2", ... index="col1", ... values="col3", ... aggregate_function=pl.element().tanh().mean(), ... ) shape: (2, 3) ┌──────┬──────────┬──────────┐ │ col1 ┆ x ┆ y │ │ --- ┆ --- ┆ --- │ │ str ┆ f64 ┆ f64 │ ╞══════╪══════════╪══════════╡ │ a ┆ 0.998347 ┆ null │ │ b ┆ 0.964028 ┆ 0.999954 │ └──────┴──────────┴──────────┘ Note that `pivot` is only available in eager mode. If you know the unique column values in advance, you can use :meth:`polars.LazyFrame.group_by` to get the same result as above in lazy mode: >>> index = pl.col("col1") >>> on = pl.col("col2") >>> values = pl.col("col3") >>> unique_column_values = ["x", "y"] >>> aggregate_function = lambda col: col.tanh().mean() >>> df.lazy().group_by(index).agg( ... aggregate_function(values.filter(on == value)).alias(value) ... for value in unique_column_values ... ).collect() # doctest: +IGNORE_RESULT shape: (2, 3) ┌──────┬──────────┬──────────┐ │ col1 ┆ x ┆ y │ │ --- ┆ --- ┆ --- │ │ str ┆ f64 ┆ f64 │ ╞══════╪══════════╪══════════╡ │ a ┆ 0.998347 ┆ null │ │ b ┆ 0.964028 ┆ 0.999954 │ └──────┴──────────┴──────────┘ Nfirstsumr+r*meanmedianlastrcountzd`aggregate_function='count'` input for `pivot` is deprecated. Please use `aggregate_function='len'`.z0.20.5rz1invalid input for `aggregate_function` argument: )rfrrrKelementr_pyexprrr+r*rrrrr&rrr pivot_expr) rrrrrr=rraggregate_exprrs rpivotDataFrame.pivot#svt (  &t4F  %d2E (# . .!W,!"!2!2!4!$ "#IJ\I_` o%  '!N/77N HH     r)r variable_name value_namecUc/O [X5nUc/O [X5nURURRXXC55$)u Unpivot a DataFrame from wide to long format. Optionally leaves identifiers set. This function is useful to massage a DataFrame into a format where one or more columns are identifier variables (index) while all other columns, considered measured variables (on), are "unpivoted" to the row axis leaving just two non-identifier columns, 'variable' and 'value'. Parameters ---------- on Column(s) or selector(s) to use as values variables; if `on` is empty all columns that are not in `index` will be used. index Column(s) or selector(s) to use as identifier variables. variable_name Name to give to the `variable` column. Defaults to "variable" value_name Name to give to the `value` column. Defaults to "value" Notes ----- If you're coming from pandas, this is similar to `pandas.DataFrame.melt`, but with `index` replacing `id_vars` and `on` replacing `value_vars`. In other frameworks, you might know this operation as `pivot_longer`. Examples -------- >>> df = pl.DataFrame( ... { ... "a": ["x", "y", "z"], ... "b": [1, 3, 5], ... "c": [2, 4, 6], ... } ... ) >>> import polars.selectors as cs >>> df.unpivot(cs.numeric(), index="a") shape: (6, 3) ┌─────┬──────────┬───────┐ │ a ┆ variable ┆ value │ │ --- ┆ --- ┆ --- │ │ str ┆ str ┆ i64 │ ╞═════╪══════════╪═══════╡ │ x ┆ b ┆ 1 │ │ y ┆ b ┆ 3 │ │ z ┆ b ┆ 5 │ │ x ┆ c ┆ 2 │ │ y ┆ c ┆ 4 │ │ z ┆ c ┆ 6 │ └─────┴──────────┴───────┘ )rfrrunpivot)rrrrrs rr DataFrame.unpivotq$sGz:R#4T#>m):4)Gtxx//:UVVrvertical)rBr fill_valuesc ^SSKnUbURU5OUnURnUS:XaUnURXx- 5n OUn URXy- 5nX-U- =m(a][ U[ 5(d$[ UR5V s/sHoPM nn URU4Sj[Xd555nUS:XaWUR[R"SX-SS9U -RS55RS5RS5nURUR!U 55S -n UV V s/sHan [ U 5HNn U R#X-U5RU R$S -['U 5R)U 5-5PMP Mc nn n [+U5$s sn fs sn n f) u Unstack a long table to a wide form without doing an aggregation. This can be much faster than a pivot, because it can skip the grouping phase. Parameters ---------- step Number of rows in the unstacked frame. how : { 'vertical', 'horizontal' } Direction of the unstack. columns Column name(s) or selector(s) to include in the operation. If set to `None` (default), use all columns. fill_values Fill values that don't fit the new size with this value. Examples -------- >>> from string import ascii_uppercase >>> df = pl.DataFrame( ... { ... "x": list(ascii_uppercase[0:8]), ... "y": pl.int_range(1, 9, eager=True), ... } ... ).with_columns( ... z=pl.int_ranges(pl.col("y"), pl.col("y") + 2, dtype=pl.UInt8), ... ) >>> df shape: (8, 3) ┌─────┬─────┬──────────┐ │ x ┆ y ┆ z │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ list[u8] │ ╞═════╪═════╪══════════╡ │ A ┆ 1 ┆ [1, 2] │ │ B ┆ 2 ┆ [2, 3] │ │ C ┆ 3 ┆ [3, 4] │ │ D ┆ 4 ┆ [4, 5] │ │ E ┆ 5 ┆ [5, 6] │ │ F ┆ 6 ┆ [6, 7] │ │ G ┆ 7 ┆ [7, 8] │ │ H ┆ 8 ┆ [8, 9] │ └─────┴─────┴──────────┘ >>> df.unstack(step=4, how="vertical") shape: (4, 6) ┌─────┬─────┬─────┬─────┬──────────┬──────────┐ │ x_0 ┆ x_1 ┆ y_0 ┆ y_1 ┆ z_0 ┆ z_1 │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ str ┆ i64 ┆ i64 ┆ list[u8] ┆ list[u8] │ ╞═════╪═════╪═════╪═════╪══════════╪══════════╡ │ A ┆ E ┆ 1 ┆ 5 ┆ [1, 2] ┆ [5, 6] │ │ B ┆ F ┆ 2 ┆ 6 ┆ [2, 3] ┆ [6, 7] │ │ C ┆ G ┆ 3 ┆ 7 ┆ [3, 4] ┆ [7, 8] │ │ D ┆ H ┆ 4 ┆ 8 ┆ [4, 5] ┆ [8, 9] │ └─────┴─────┴─────┴─────┴──────────┴──────────┘ >>> df.unstack(step=2, how="horizontal") shape: (4, 6) ┌─────┬─────┬─────┬─────┬──────────┬──────────┐ │ x_0 ┆ x_1 ┆ y_0 ┆ y_1 ┆ z_0 ┆ z_1 │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ str ┆ i64 ┆ i64 ┆ list[u8] ┆ list[u8] │ ╞═════╪═════╪═════╪═════╪══════════╪══════════╡ │ A ┆ B ┆ 1 ┆ 2 ┆ [1, 2] ┆ [2, 3] │ │ C ┆ D ┆ 3 ┆ 4 ┆ [3, 4] ┆ [4, 5] │ │ E ┆ F ┆ 5 ┆ 6 ┆ [5, 6] ┆ [6, 7] │ │ G ┆ H ┆ 7 ┆ 8 ┆ [7, 8] ┆ [8, 9] │ └─────┴─────┴─────┴─────┴──────────┴──────────┘ >>> import polars.selectors as cs >>> df.unstack(step=5, columns=cs.numeric(), fill_values=0) shape: (5, 2) ┌─────┬─────┐ │ y_0 ┆ y_1 │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 6 │ │ 2 ┆ 7 │ │ 3 ┆ 8 │ │ 4 ┆ 0 │ │ 5 ┆ 0 │ └─────┴─────┘ rNr c3L># UHupURUT5v M g7fr)r)rqrd next_filln_fills rrs$DataFrame.unstack..%s*$8LA!!)V44$8s!$r@T)rF __sort_orderrr)mathrJrceilrrrYrr#r]rK int_rangerZr@rbr_log10rarrzfillr)rsteprBrr rrrrn_colsr zfill_valrd slice_nbrslicesrs @runstackDataFrame.unstack$sv %,%8T[[ !d * FYYv/FFYYv/F_v- -6 -k40049"((ODOq{O D$'$8B , [[FO4@6IPP& n%n% JJtzz&12Q6   "6] GGI& / 5 5 s9~33I>> +      7E& s  F? A(G)r= include_keyas_dictcgrrrr`r=rr rAs rrSDataFrame.partition_by9%sr)r=rcgrrr"s rrSr#C%s,/rcgrrr"s rrSr#M%s>Arc[X/UQ76nURRXbU5Vs/sHnURU5PM nnU(aU(a0UV s/sH"oR U5R S5PM$ n n O@U(d Sn [ U 5eUR U5RSS9R5n [[X55$U$s snfs sn f)uZ Group by the given columns and return the groups as separate dataframes. Parameters ---------- by Column name(s) or selector(s) to group by. *more_by Additional names of columns to group by, specified as positional arguments. maintain_order Ensure that the order of the groups is consistent with the input data. This is slower than a default partition by operation. include_key Include the columns used to partition the DataFrame in the output. as_dict Return a dictionary instead of a list. The dictionary keys are tuples of the distinct group values that identify each group. Examples -------- Pass a single column name to partition by that column. >>> df = pl.DataFrame( ... { ... "a": ["a", "b", "a", "b", "c"], ... "b": [1, 2, 1, 3, 3], ... "c": [5, 4, 3, 2, 1], ... } ... ) >>> df.partition_by("a") # doctest: +IGNORE_RESULT [shape: (2, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ a ┆ 1 ┆ 5 │ │ a ┆ 1 ┆ 3 │ └─────┴─────┴─────┘, shape: (2, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ b ┆ 2 ┆ 4 │ │ b ┆ 3 ┆ 2 │ └─────┴─────┴─────┘, shape: (1, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ c ┆ 3 ┆ 1 │ └─────┴─────┴─────┘] Partition by multiple columns by either passing a list of column names, or by specifying each column name as a positional argument. >>> df.partition_by("a", "b") # doctest: +IGNORE_RESULT [shape: (2, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ a ┆ 1 ┆ 5 │ │ a ┆ 1 ┆ 3 │ └─────┴─────┴─────┘, shape: (1, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ b ┆ 2 ┆ 4 │ └─────┴─────┴─────┘, shape: (1, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ b ┆ 3 ┆ 2 │ └─────┴─────┴─────┘, shape: (1, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ c ┆ 3 ┆ 1 │ └─────┴─────┴─────┘] Return the partitions as a dictionary by specifying `as_dict=True`. >>> import polars.selectors as cs >>> df.partition_by(cs.string(), as_dict=True) # doctest: +IGNORE_RESULT {('a',): shape: (2, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ a ┆ 1 ┆ 5 │ │ a ┆ 1 ┆ 3 │ └─────┴─────┴─────┘, ('b',): shape: (2, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ b ┆ 2 ┆ 4 │ │ b ┆ 3 ┆ 2 │ └─────┴─────┴─────┘, ('c',): shape: (1, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ c ┆ 3 ┆ 1 │ └─────┴─────┴─────┘} rzVcannot use `partition_by` with `maintain_order=False, include_key=False, as_dict=True`Tr) rfrrSrrJrruniquer-rr#) rr`r=rr rA by_parsedr partitionsprrs rrSr#W%sL&d99 xx,,Y T T OOC T  =GHZ),003ZH%rC$S/) I.55T5JOOQE./ /! Is C)C! fill_valuec~SSKJn UR5RXS9R UR 5S9$)u Shift values by the given number of indices. Parameters ---------- n Number of indices to shift forward. If a negative value is passed, values are shifted in the opposite direction instead. fill_value Fill the resulting null values with this value. Accepts scalar expression input. Non-expression inputs are parsed as literals. Notes ----- This method is similar to the `LAG` operation in SQL when the value for `n` is positive. With a negative value for `n`, it is similar to `LEAD`. Examples -------- By default, values are shifted forward by one index. >>> df = pl.DataFrame( ... { ... "a": [1, 2, 3, 4], ... "b": [5, 6, 7, 8], ... } ... ) >>> df.shift() shape: (4, 2) ┌──────┬──────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞══════╪══════╡ │ null ┆ null │ │ 1 ┆ 5 │ │ 2 ┆ 6 │ │ 3 ┆ 7 │ └──────┴──────┘ Pass a negative value to shift in the opposite direction instead. >>> df.shift(-2) shape: (4, 2) ┌──────┬──────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞══════╪══════╡ │ 3 ┆ 7 │ │ 4 ┆ 8 │ │ null ┆ null │ │ null ┆ null │ └──────┴──────┘ Specify `fill_value` to fill the resulting null values. >>> df.shift(-2, fill_value=100) shape: (4, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 3 ┆ 7 │ │ 4 ┆ 8 │ │ 100 ┆ 100 │ │ 100 ┆ 100 │ └─────┴─────┘ rrr+r)rrrshiftrr)rrQr,rs rr.DataFrame.shift%s9N = IIK U1U , W=#7#7#9W : rcH[URR55$)u Get a mask of all duplicated rows in this DataFrame. Examples -------- >>> df = pl.DataFrame( ... { ... "a": [1, 2, 3, 1], ... "b": ["x", "y", "z", "x"], ... } ... ) >>> df.is_duplicated() shape: (4,) Series: '' [bool] [ true false false true ] This mask can be used to visualize the duplicated lines like this: >>> df.filter(df.is_duplicated()) shape: (2, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ str │ ╞═════╪═════╡ │ 1 ┆ x │ │ 1 ┆ x │ └─────┴─────┘ )r9r is_duplicatedr s rr1DataFrame.is_duplicated@&sFdhh,,.//rcH[URR55$)up Get a mask of all unique rows in this DataFrame. Examples -------- >>> df = pl.DataFrame( ... { ... "a": [1, 2, 3, 1], ... "b": ["x", "y", "z", "x"], ... } ... ) >>> df.is_unique() shape: (4,) Series: '' [bool] [ false true true false ] This mask can be used to visualize the unique lines like this: >>> df.filter(df.is_unique()) shape: (2, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ str │ ╞═════╪═════╡ │ 2 ┆ y │ │ 3 ┆ z │ └─────┴─────┘ )r9r is_uniquer s rr4DataFrame.is_uniquee&sFdhh((*++rcH[URR55$)aS Start a lazy query from this point. This returns a `LazyFrame` object. Operations on a `LazyFrame` are not executed until this is triggered by calling one of: * :meth:`.collect() ` (run on all data) * :meth:`.explain() ` (print the query plan) * :meth:`.show_graph() ` (show the query plan as graphviz graph) * :meth:`.collect_schema() ` (return the final frame schema) Lazy operations are recommended because they allow for query optimization and additional parallelism. Returns ------- LazyFrame Examples -------- >>> df = pl.DataFrame( ... { ... "a": [None, 2, 3, 4], ... "b": [0.5, None, 2.5, 13], ... "c": [True, True, False, None], ... } ... ) >>> df.lazy() )r8rrr s rrDataFrame.lazy&sF ((rcSSKJn UR5R"U0UD6R UR 5S9$)u Select columns from this DataFrame. Parameters ---------- *exprs Column(s) to select, specified as positional arguments. Accepts expression input. Strings are parsed as column names, other non-expression inputs are parsed as literals. **named_exprs Additional columns to select, specified as keyword arguments. The columns will be renamed to the keyword used. Examples -------- Pass the name of a column to select that column. >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.select("foo") shape: (3, 1) ┌─────┐ │ foo │ │ --- │ │ i64 │ ╞═════╡ │ 1 │ │ 2 │ │ 3 │ └─────┘ Multiple columns can be selected by passing a list of column names. >>> df.select(["foo", "bar"]) shape: (3, 2) ┌─────┬─────┐ │ foo ┆ bar │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 6 │ │ 2 ┆ 7 │ │ 3 ┆ 8 │ └─────┴─────┘ Multiple columns can also be selected using positional arguments instead of a list. Expressions are also accepted. >>> df.select(pl.col("foo"), pl.col("bar") + 1) shape: (3, 2) ┌─────┬─────┐ │ foo ┆ bar │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 7 │ │ 2 ┆ 8 │ │ 3 ┆ 9 │ └─────┴─────┘ Use keyword arguments to easily name your expression inputs. >>> df.select(threshold=pl.when(pl.col("foo") > 2).then(10).otherwise(0)) shape: (3, 1) ┌───────────┐ │ threshold │ │ --- │ │ i32 │ ╞═══════════╡ │ 0 │ │ 0 │ │ 10 │ └───────────┘ rrr)rrrrJrrrexprs named_exprsrs rrJDataFrame.select&sGd = IIK V  +) + W=#7#7#9W : rcSSKJn UR5R"U0UD6R UR 5S9$)a  Select columns from this DataFrame. This will run all expression sequentially instead of in parallel. Use this when the work per expression is cheap. Parameters ---------- *exprs Column(s) to select, specified as positional arguments. Accepts expression input. Strings are parsed as column names, other non-expression inputs are parsed as literals. **named_exprs Additional columns to select, specified as keyword arguments. The columns will be renamed to the keyword used. See Also -------- select rrr)rrr select_seqrrr9s rr>DataFrame.select_seq 'sF. = IIK Z  /"- / W=#7#7#9W : rcSSKJn UR5R"U0UD6R UR 5S9$)u? Add columns to this DataFrame. Added columns will replace existing columns with the same name. Parameters ---------- *exprs Column(s) to add, specified as positional arguments. Accepts expression input. Strings are parsed as column names, other non-expression inputs are parsed as literals. **named_exprs Additional columns to add, specified as keyword arguments. The columns will be renamed to the keyword used. Returns ------- DataFrame A new DataFrame with the columns added. Notes ----- Creating a new DataFrame using this method does not create a new copy of existing data. Examples -------- Pass an expression to add it as a new column. >>> df = pl.DataFrame( ... { ... "a": [1, 2, 3, 4], ... "b": [0.5, 4, 10, 13], ... "c": [True, True, False, True], ... } ... ) >>> df.with_columns((pl.col("a") ** 2).alias("a^2")) shape: (4, 4) ┌─────┬──────┬───────┬─────┐ │ a ┆ b ┆ c ┆ a^2 │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ bool ┆ i64 │ ╞═════╪══════╪═══════╪═════╡ │ 1 ┆ 0.5 ┆ true ┆ 1 │ │ 2 ┆ 4.0 ┆ true ┆ 4 │ │ 3 ┆ 10.0 ┆ false ┆ 9 │ │ 4 ┆ 13.0 ┆ true ┆ 16 │ └─────┴──────┴───────┴─────┘ Added columns will replace existing columns with the same name. >>> df.with_columns(pl.col("a").cast(pl.Float64)) shape: (4, 3) ┌─────┬──────┬───────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ bool │ ╞═════╪══════╪═══════╡ │ 1.0 ┆ 0.5 ┆ true │ │ 2.0 ┆ 4.0 ┆ true │ │ 3.0 ┆ 10.0 ┆ false │ │ 4.0 ┆ 13.0 ┆ true │ └─────┴──────┴───────┘ Multiple columns can be added using positional arguments. >>> df.with_columns( ... (pl.col("a") ** 2).alias("a^2"), ... (pl.col("b") / 2).alias("b/2"), ... (pl.col("c").not_()).alias("not c"), ... ) shape: (4, 6) ┌─────┬──────┬───────┬─────┬──────┬───────┐ │ a ┆ b ┆ c ┆ a^2 ┆ b/2 ┆ not c │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ bool ┆ i64 ┆ f64 ┆ bool │ ╞═════╪══════╪═══════╪═════╪══════╪═══════╡ │ 1 ┆ 0.5 ┆ true ┆ 1 ┆ 0.25 ┆ false │ │ 2 ┆ 4.0 ┆ true ┆ 4 ┆ 2.0 ┆ false │ │ 3 ┆ 10.0 ┆ false ┆ 9 ┆ 5.0 ┆ true │ │ 4 ┆ 13.0 ┆ true ┆ 16 ┆ 6.5 ┆ false │ └─────┴──────┴───────┴─────┴──────┴───────┘ Multiple columns can also be added by passing a list of expressions. >>> df.with_columns( ... [ ... (pl.col("a") ** 2).alias("a^2"), ... (pl.col("b") / 2).alias("b/2"), ... (pl.col("c").not_()).alias("not c"), ... ] ... ) shape: (4, 6) ┌─────┬──────┬───────┬─────┬──────┬───────┐ │ a ┆ b ┆ c ┆ a^2 ┆ b/2 ┆ not c │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ bool ┆ i64 ┆ f64 ┆ bool │ ╞═════╪══════╪═══════╪═════╪══════╪═══════╡ │ 1 ┆ 0.5 ┆ true ┆ 1 ┆ 0.25 ┆ false │ │ 2 ┆ 4.0 ┆ true ┆ 4 ┆ 2.0 ┆ false │ │ 3 ┆ 10.0 ┆ false ┆ 9 ┆ 5.0 ┆ true │ │ 4 ┆ 13.0 ┆ true ┆ 16 ┆ 6.5 ┆ false │ └─────┴──────┴───────┴─────┴──────┴───────┘ Use keyword arguments to easily name your expression inputs. >>> df.with_columns( ... ab=pl.col("a") * pl.col("b"), ... not_c=pl.col("c").not_(), ... ) shape: (4, 5) ┌─────┬──────┬───────┬──────┬───────┐ │ a ┆ b ┆ c ┆ ab ┆ not_c │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ bool ┆ f64 ┆ bool │ ╞═════╪══════╪═══════╪══════╪═══════╡ │ 1 ┆ 0.5 ┆ true ┆ 0.5 ┆ false │ │ 2 ┆ 4.0 ┆ true ┆ 8.0 ┆ false │ │ 3 ┆ 10.0 ┆ false ┆ 30.0 ┆ true │ │ 4 ┆ 13.0 ┆ true ┆ 52.0 ┆ false │ └─────┴──────┴───────┴──────┴───────┘ rrr)rrrr]rrr9s rr]DataFrame.with_columns('sG~ = IIK \   1$/ 1 W=#7#7#9W : rcSSKJn UR5R"U0UD6R UR 5S9$)a Add columns to this DataFrame. Added columns will replace existing columns with the same name. This will run all expression sequentially instead of in parallel. Use this when the work per expression is cheap. Parameters ---------- *exprs Column(s) to add, specified as positional arguments. Accepts expression input. Strings are parsed as column names, other non-expression inputs are parsed as literals. **named_exprs Additional columns to add, specified as keyword arguments. The columns will be renamed to the keyword used. Returns ------- DataFrame A new DataFrame with the columns added. See Also -------- with_columns rrr)rrrwith_columns_seqrrr9s rrCDataFrame.with_columns_seq'sH@ = IIK   $ 5(3 5 W=#7#7#9W : rcgrrrrs rn_chunksDataFrame.n_chunks'sADrcgrrrFs rrGrH's?BrcUS:XaURR5$US:Xa.UR5Vs/sHo"R5PM sn$SU<S3n[U5es snf)a Get number of chunks used by the ChunkedArrays of this DataFrame. Parameters ---------- strategy : {'first', 'all'} Return the number of chunks of the 'first' column, or 'all' columns in this DataFrame. Examples -------- >>> df = pl.DataFrame( ... { ... "a": [1, 2, 3, 4], ... "b": [0.5, 4, 10, 13], ... "c": [True, True, False, True], ... } ... ) >>> df.n_chunks() 1 >>> df.n_chunks(strategy="all") [1, 1, 1] rrLz!unexpected input for `strategy`: z Choose one of {'first', 'all'})rrGrr)rrrdrs rrGrH'sm2 w 88$$& &  *.--/:/QJJL/: :4H<68 S/ ! ;sA'cSSKJn UR5R5R UR 5S9$)u Aggregate the columns of this DataFrame to their maximum value. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.max() shape: (1, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 3 ┆ 8 ┆ c │ └─────┴─────┴─────┘ rrr)rrrr+rrrrs rr+ DataFrame.max(1. =yy{ ((}7K7K7M(NNrcUR[R"[R"55S9R 5$)aN Get the maximum value horizontally across columns. Returns ------- Series A Series named `"max"`. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [4.0, 5.0, 6.0], ... } ... ) >>> df.max_horizontal() shape: (3,) Series: 'max' [f64] [ 4.0 5.0 6.0 ] )r+)rJrKmax_horizontalrLr r s rrPDataFrame.max_horizontal(/4{{q//8{9CCEErcSSKJn UR5R5R UR 5S9$)u Aggregate the columns of this DataFrame to their minimum value. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.min() shape: (1, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ └─────┴─────┴─────┘ rrr)rrrr*rrrLs rr* DataFrame.min8(rNrcUR[R"[R"55S9R 5$)aN Get the minimum value horizontally across columns. Returns ------- Series A Series named `"min"`. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [4.0, 5.0, 6.0], ... } ... ) >>> df.min_horizontal() shape: (3,) Series: 'min' [f64] [ 1.0 2.0 3.0 ] )r*)rJrKmin_horizontalrLr r s rrVDataFrame.min_horizontalS(rRrcSSKJn UR5R5R UR 5S9$)u' Aggregate the columns of this DataFrame to their sum value. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.sum() shape: (1, 3) ┌─────┬─────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪══════╡ │ 6 ┆ 21 ┆ null │ └─────┴─────┴──────┘ rrr)rrrrrrrLs rr DataFrame.sumo(rNr ignore_nullscUR[R"[R"5US9S9R 5$)a Sum all values horizontally across columns. Parameters ---------- ignore_nulls Ignore null values (default). If set to `False`, any null value in the input will lead to a null output. Returns ------- Series A Series named `"sum"`. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [4.0, 5.0, 6.0], ... } ... ) >>> df.sum_horizontal() shape: (3,) Series: 'sum' [f64] [ 5.0 7.0 9.0 ] rZ)r)rJrKsum_horizontalrLr rr[s rr]DataFrame.sum_horizontal(s8@{{  |D )+ rcSSKJn UR5R5R UR 5S9$)u Aggregate the columns of this DataFrame to their mean value. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... "spam": [True, False, None], ... } ... ) >>> df.mean() shape: (1, 4) ┌─────┬─────┬──────┬──────┐ │ foo ┆ bar ┆ ham ┆ spam │ │ --- ┆ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ str ┆ f64 │ ╞═════╪═════╪══════╪══════╡ │ 2.0 ┆ 7.0 ┆ null ┆ 0.5 │ └─────┴─────┴──────┴──────┘ rrr)rrrrrrrLs rrDataFrame.mean(s30 =yy{!)) 8L8L8N)OOrcUR[R"[R"5US9S9R 5$)a Take the mean of all values horizontally across columns. Parameters ---------- ignore_nulls Ignore null values (default). If set to `False`, any null value in the input will lead to a null output. Returns ------- Series A Series named `"mean"`. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [4.0, 5.0, 6.0], ... } ... ) >>> df.mean_horizontal() shape: (3,) Series: 'mean' [f64] [ 2.5 3.5 4.5 ] rZ)r)rJrKmean_horizontalrLr r^s rrcDataFrame.mean_horizontal(s8@{{""1557F )+ rcSSKJn UR5RU5R UR 5S9$)u Aggregate the columns of this DataFrame to their standard deviation value. Parameters ---------- ddof “Delta Degrees of Freedom”: the divisor used in the calculation is N - ddof, where N represents the number of elements. By default ddof is 1. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.std() shape: (1, 3) ┌─────┬─────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ str │ ╞═════╪═════╪══════╡ │ 1.0 ┆ 1.0 ┆ null │ └─────┴─────┴──────┘ >>> df.std(ddof=0) shape: (1, 3) ┌──────────┬──────────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ str │ ╞══════════╪══════════╪══════╡ │ 0.816497 ┆ 0.816497 ┆ null │ └──────────┴──────────┴──────┘ rrr)rrrstdrrrddofrs rrf DataFrame.std(4N =yy{t$,,=;O;O;Q,RRrcSSKJn UR5RU5R UR 5S9$)u Aggregate the columns of this DataFrame to their variance value. Parameters ---------- ddof “Delta Degrees of Freedom”: the divisor used in the calculation is N - ddof, where N represents the number of elements. By default ddof is 1. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.var() shape: (1, 3) ┌─────┬─────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ str │ ╞═════╪═════╪══════╡ │ 1.0 ┆ 1.0 ┆ null │ └─────┴─────┴──────┘ >>> df.var(ddof=0) shape: (1, 3) ┌──────────┬──────────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ str │ ╞══════════╪══════════╪══════╡ │ 0.666667 ┆ 0.666667 ┆ null │ └──────────┴──────────┴──────┘ rrr)rrrvarrrrgs rrl DataFrame.var)rjrcSSKJn UR5R5R UR 5S9$)u- Aggregate the columns of this DataFrame to their median value. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.median() shape: (1, 3) ┌─────┬─────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ str │ ╞═════╪═════╪══════╡ │ 2.0 ┆ 7.0 ┆ null │ └─────┴─────┴──────┘ rrr)rrrrrrrLs rrDataFrame.medianD)s3. =yy{!!#++-:N:N:P+QQrc/nURR5Hup#UR5(d[U[5(a5UR [ R"U5R55 MdUR [ R"S5RU55 M URU5$)u2 Aggregate the columns of this DataFrame to their product values. Examples -------- >>> df = pl.DataFrame( ... { ... "a": [1, 2, 3], ... "b": [0.5, 4, 10], ... "c": [True, True, False], ... } ... ) >>> df.product() shape: (1, 3) ┌─────┬──────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ i64 │ ╞═════╪══════╪═════╡ │ 6 ┆ 20.0 ┆ 0 │ └─────┴──────┴─────┘ N) rra is_numericrr@rrKraproductrbrZrJ)rr:rrs rrrDataFrame.product_)s0 ))+HD}}*R"9"9 QUU4[0023 QUU4[..t45 , {{5!!rcSSKJn UR5RX5R UR 5S9$)u Aggregate the columns of this DataFrame to their quantile value. Parameters ---------- quantile Quantile between 0.0 and 1.0. interpolation : {'nearest', 'higher', 'lower', 'midpoint', 'linear', 'equiprobable'} Interpolation method. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.quantile(0.5, "nearest") shape: (1, 3) ┌─────┬─────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ str │ ╞═════╪═════╪══════╡ │ 2.0 ┆ 7.0 ┆ null │ └─────┴─────┴──────┘ rrr)rrrquantilerr)rrur0rs rruDataFrame.quantile)s7@ = IIK Xh . W=#7#7#9W : r)r drop_firstructUb [X5nURURRXX455$)u Convert categorical variables into dummy/indicator variables. Parameters ---------- columns Column name(s) or selector(s) that should be converted to dummy variables. If set to `None` (default), convert all columns. separator Separator/delimiter used when generating column names. drop_first Remove the first category from the variables being encoded. drop_nulls If there are `None` values in the series, a `null` column is not generated Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2], ... "bar": [3, 4], ... "ham": ["a", "b"], ... } ... ) >>> df.to_dummies() shape: (2, 6) ┌───────┬───────┬───────┬───────┬───────┬───────┐ │ foo_1 ┆ foo_2 ┆ bar_3 ┆ bar_4 ┆ ham_a ┆ ham_b │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ u8 ┆ u8 ┆ u8 ┆ u8 ┆ u8 ┆ u8 │ ╞═══════╪═══════╪═══════╪═══════╪═══════╪═══════╡ │ 1 ┆ 0 ┆ 1 ┆ 0 ┆ 1 ┆ 0 │ │ 0 ┆ 1 ┆ 0 ┆ 1 ┆ 0 ┆ 1 │ └───────┴───────┴───────┴───────┴───────┴───────┘ >>> df.to_dummies(drop_first=True) shape: (2, 3) ┌───────┬───────┬───────┐ │ foo_2 ┆ bar_4 ┆ ham_b │ │ --- ┆ --- ┆ --- │ │ u8 ┆ u8 ┆ u8 │ ╞═══════╪═══════╪═══════╡ │ 0 ┆ 0 ┆ 0 │ │ 1 ┆ 1 ┆ 1 │ └───────┴───────┴───────┘ >>> import polars.selectors as cs >>> df.to_dummies(cs.integer(), separator=":") shape: (2, 5) ┌───────┬───────┬───────┬───────┬─────┐ │ foo:1 ┆ foo:2 ┆ bar:3 ┆ bar:4 ┆ ham │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ u8 ┆ u8 ┆ u8 ┆ u8 ┆ str │ ╞═══════╪═══════╪═══════╪═══════╪═════╡ │ 1 ┆ 0 ┆ 1 ┆ 0 ┆ a │ │ 0 ┆ 1 ┆ 0 ┆ 1 ┆ b │ └───────┴───────┴───────┴───────┴─────┘ >>> df.to_dummies(cs.integer(), drop_first=True, separator=":") shape: (2, 3) ┌───────┬───────┬─────┐ │ foo:2 ┆ bar:4 ┆ ham │ │ --- ┆ --- ┆ --- │ │ u8 ┆ u8 ┆ str │ ╞═══════╪═══════╪═════╡ │ 0 ┆ 0 ┆ a │ │ 1 ┆ 1 ┆ b │ └───────┴───────┴─────┘ )rfrr to_dummies)rrrrwrus rryDataFrame.to_dummies)s:Z  '6G HH  J K  rany)rr=cSSKJn UR5RXUS9R UR 5S9$)u Drop duplicate rows from this dataframe. Parameters ---------- subset Column name(s) or selector(s), to consider when identifying duplicate rows. If set to `None` (default), use all columns. keep : {'first', 'last', 'any', 'none'} Which of the duplicate rows to keep. * 'any': Does not give any guarantee of which row is kept. This allows more optimizations. * 'none': Don't keep duplicate rows. * 'first': Keep first unique row. * 'last': Keep last unique row. maintain_order Keep the same order as the original DataFrame. This is more expensive to compute. Settings this to `True` blocks the possibility to run on the streaming engine. Returns ------- DataFrame DataFrame with unique rows. Warnings -------- This method will fail if there is a column of type `List` in the DataFrame or subset. Notes ----- If you're coming from pandas, this is similar to `pandas.DataFrame.drop_duplicates`. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3, 1], ... "bar": ["a", "a", "a", "a"], ... "ham": ["b", "b", "b", "b"], ... } ... ) >>> df.unique(maintain_order=True) shape: (3, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ str ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ a ┆ b │ │ 2 ┆ a ┆ b │ │ 3 ┆ a ┆ b │ └─────┴─────┴─────┘ >>> df.unique(subset=["bar", "ham"], maintain_order=True) shape: (1, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ str ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ a ┆ b │ └─────┴─────┴─────┘ >>> df.unique(keep="last", maintain_order=True) shape: (3, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ str ┆ str │ ╞═════╪═════╪═════╡ │ 2 ┆ a ┆ b │ │ 3 ┆ a ┆ b │ │ 1 ┆ a ┆ b │ └─────┴─────┴─────┘ rr)rrrr=r)rrrr'rr)rrrrr=rs rr'DataFrame.unique)s;j = IIK V6^V L W=#7#7#9W : rch[U[5(a[R"U5nO[U[R 5(aUnOl[U[ 5(a'[U5S:Xa[[US55nO0Uc[R"5OUn[R"U5nSSK J n UR5RUR!55R#UR%5S9nUR'5(aS$UR)S5S$)a2 Return the number of unique rows, or the number of unique row-subsets. Parameters ---------- subset One or more columns/expressions that define what to count; omit to return the count of unique rows. Notes ----- This method operates at the `DataFrame` level; to operate on subsets at the expression level you can make use of struct-packing instead, for example: >>> expr_unique_subset = pl.struct("a", "b").n_unique() If instead you want to count the number of unique values per-column, you can also use expression-level syntax to return a new frame containing that result: >>> df = pl.DataFrame( ... [[1, 2, 3], [1, 2, 4]], schema=["a", "b", "c"], orient="row" ... ) >>> df_nunique = df.select(pl.all().n_unique()) In aggregate context there is also an equivalent method for returning the unique values per-group: >>> df_agg_nunique = df.group_by("a").n_unique() Examples -------- >>> df = pl.DataFrame( ... { ... "a": [1, 1, 2, 3, 4, 5], ... "b": [0.5, 0.5, 1.0, 2.0, 3.0, 3.0], ... "c": [True, True, True, False, True, True], ... } ... ) >>> df.n_unique() 5 Simple columns subset. >>> df.n_unique(subset=["b", "c"]) 4 Expression subset. >>> df.n_unique( ... subset=[ ... (pl.col("a") // 2), ... (pl.col("c") | (pl.col("b") >= 2)), ... ], ... ) 3 rrrr)rrrKrarrsrrr7r)rLr6rrrrJn_uniquerrr5r)rrrrR struct_fieldsrrs rrDataFrame.n_uniqueX*sr fc " "55=D  ( (D  ) )c&kQ.>26!9=>D(.AEEGVM88M*D< IIK VDMMO $ W=#7#7#9W : KKMMq3rvvay|3rz\`DataFrame.approx_n_unique` is deprecated; use `select(pl.all().approx_n_unique())` instead.cSSKJn UR5R5R UR 5S9$)uN Approximate count of unique values. .. deprecated:: 0.20.11 Use the `select(pl.all().approx_n_unique())` method instead. This is done using the HyperLogLog++ algorithm for cardinality estimation. Examples -------- >>> df = pl.DataFrame( ... { ... "a": [1, 2, 3, 4], ... "b": [1, 2, 1, 1], ... } ... ) >>> df.approx_n_unique() # doctest: +SKIP shape: (1, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ u32 ┆ u32 │ ╞═════╪═════╡ │ 4 ┆ 2 │ └─────┴─────┘ rrr)rrrapprox_n_uniquerrrLs rrDataFrame.approx_n_unique*s7> = IIK ' ' ) 1 1 @T@T@V 1 W rcTURURR55$)z Rechunk the data in this DataFrame to a contiguous allocation. This will make sure all subsequent operations have optimal and predictable performance. )rrrr s rrDataFrame.rechunk*s txx//122rcTURURR55$)u) Create a new DataFrame that shows the null counts per column. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, None, 3], ... "bar": [6, 7, None], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.null_count() shape: (1, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ u32 ┆ u32 ┆ u32 │ ╞═════╪═════╪═════╡ │ 1 ┆ 1 ┆ 0 │ └─────┴─────┴─────┘ )rr null_countr s rrDataFrame.null_count*s .txx22455r)fractionwith_replacementshuffleseedc&UbUb Sn[U5eUc[R"SS5nUcpUbm[U[R 5(d[R "SU/5nUR URRURX4U55$UcSn[U[R 5(d[R "SU/5nUR URRURX4U55$)u} Sample from this DataFrame. Parameters ---------- n Number of items to return. Cannot be used with `fraction`. Defaults to 1 if `fraction` is None. fraction Fraction of items to return. Cannot be used with `n`. with_replacement Allow values to be sampled more than once. shuffle If set to True, the order of the sampled rows will be shuffled. If set to False (default), the order of the returned rows will be neither stable nor fully random. seed Seed for the random number generator. If set to None (default), a random seed is generated for each sample operation. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.sample(n=2, seed=0) # doctest: +IGNORE_RESULT shape: (2, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 3 ┆ 8 ┆ c │ │ 2 ┆ 7 ┆ b │ └─────┴─────┴─────┘ z&cannot specify both `n` and `fraction`r'fracrr) rrandomrandintrrrurr sample_fracrsample_n)rrQrrrrrs rsampleDataFrame.sample*sb =X1:CS/ ! <>>!U+D 9-h 2299VhZ8??$$X[[2BTR  9A!RYY'' "qc"Atxx007GRVWXXrcURS5n[SUR5HnU"X RU55nM U$)a3 Apply a horizontal reduction on a DataFrame. This can be used to effectively determine aggregations on a row level, and can be applied to any DataType that can be supercast (cast to a similar parent type). An example of the supercast rules when applying an arithmetic operation on two DataTypes are for instance: - Int8 + String = String - Float32 + Int64 = Float32 - Float32 + Float64 = Float64 Examples -------- A horizontal sum operation: >>> df = pl.DataFrame( ... { ... "a": [2, 1, 3], ... "b": [1, 2, 3], ... "c": [1.0, 2.0, 3.0], ... } ... ) >>> df.fold(lambda s1, s2: s1 + s2) shape: (3,) Series: 'a' [f64] [ 4.0 5.0 9.0 ] A horizontal minimum operation: >>> df = pl.DataFrame({"a": [2, 1, 3], "b": [1, 2, 3], "c": [1.0, 2.0, 3.0]}) >>> df.fold(lambda s1, s2: s1.zip_with(s1 < s2, s2)) shape: (3,) Series: 'a' [f64] [ 1.0 1.0 3.0 ] A horizontal string concatenation: >>> df = pl.DataFrame( ... { ... "a": ["foo", "bar", None], ... "b": [1, 2, 3], ... "c": [1.0, 2.0, 3.0], ... } ... ) >>> df.fold(lambda s1, s2: s1 + s2) shape: (3,) Series: 'a' [str] [ "foo11.0" "bar22.0" null ] A horizontal boolean or, similar to a row-wise .any(): >>> df = pl.DataFrame( ... { ... "a": [False, False, True], ... "b": [False, True, False], ... } ... ) >>> df.fold(lambda s1, s2: s1 | s2) shape: (3,) Series: 'a' [bool] [ false true true ] Parameters ---------- operation function that takes two `Series` and returns a `Series`. rr)r rYr)r operationaccres rfoldDataFrame.fold3+s@nnnQq$**%AC!23C& r) by_predicater,cgrrrrrr,s rr DataFrame.row+sr)rcgrrrs rrr+srcUbUb Sn[U5e[U[R5(a Sn[ U5eUbCUR R U5nU(a[[URU55$U$Ub[U[R5(dS[U5<3n[ U5eURU5R5n[U5nUS:aSU<SUS3n[U5eUS:XaSU<S 3n[U5eUSnU(a[[URU55$U$S n[U5e) a Get the values of a single row, either by index or by predicate. Parameters ---------- index Row index. by_predicate Select the row according to a given expression/predicate. named Return a dictionary instead of a tuple. The dictionary is a mapping of column name to row value. This is more expensive than returning a regular tuple, but allows for accessing values by column name. Returns ------- tuple (default) or dictionary of row values Notes ----- The `index` and `by_predicate` params are mutually exclusive. Additionally, to ensure clarity, the `by_predicate` parameter must be supplied by keyword. When using `by_predicate` it is an error condition if anything other than one row is returned; more than one row raises `TooManyRowsReturnedError`, and zero rows will raise `NoRowsReturnedError` (both inherit from `RowsError`). Warnings -------- You should NEVER use this method to iterate over a DataFrame; if you require row-iteration you should strongly prefer use of `iter_rows()` instead. See Also -------- iter_rows : Row iterator over frame data (does not materialise all rows). rows : Materialise all frame data as a list of rows (potentially expensive). item: Return dataframe element as a scalar. Examples -------- Specify an index to return the row at the given index as a tuple. >>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.row(2) (3, 8, 'c') Specify `named=True` to get a dictionary instead with a mapping of column names to row values. >>> df.row(2, named=True) {'foo': 3, 'bar': 8, 'ham': 'c'} Use `by_predicate` to return the row that matches the given predicate. >>> df.row(by_predicate=(pl.col("ham") == "b")) (2, 7, 'b') z>cannot set both 'index' and 'by_predicate'; mutually exclusivez returned z rowsrz> returned no rowsz,one of `index` or `by_predicate` must be set)rrrrsrr row_tuplerr#rr3rr-rr`r_)rrrr,rrr-rs rrr+sKL  !9RCS/ ! rww ' 'PCC.  (($$U+CC c233  %lBGG44IJ]^jJkInon$;;|,113DYFz#L#3;vheL.s331#L#33EF)#..q'CC c233 @CS/ !rr+cgrrrr,s rr-DataFrame.rows ,r&rcgrrrs rr-r,sEHrc U(aM[[URpCnURR 5Vs/sHoR"U"XE55PM sn$URR 5$s snf)u Returns all data in the DataFrame as a list of rows of python-native values. By default, each row is returned as a tuple of values given in the same order as the frame columns. Setting `named=True` will return rows of dictionaries instead. Parameters ---------- named Return dictionaries instead of tuples. The dictionaries are a mapping of column name to row value. This is more expensive than returning a regular tuple, but allows for accessing values by column name. Notes ----- If you have `ns`-precision temporal values you should be aware that Python natively only supports up to `μs`-precision; `ns`-precision values will be truncated to microseconds on conversion to Python. If this matters to your use-case you should export to a different format (such as Arrow or NumPy). Warnings -------- Row-iteration is not optimal as the underlying data is stored in columnar form; where possible, prefer export via one of the dedicated export/output methods. You should also consider using `iter_rows` instead, to avoid materialising all the data at once; there is little performance difference between the two, but peak memory can be reduced if processing rows in batches. Returns ------- list of row value tuples (default), or list of dictionaries (if `named=True`). See Also -------- iter_rows : Row iterator over frame data (does not materialise all rows). rows_by_key : Materialises frame data as a key-indexed dictionary. Examples -------- >>> df = pl.DataFrame( ... { ... "x": ["a", "b", "b", "a"], ... "y": [1, 2, 3, 4], ... "z": [0, 3, 6, 9], ... } ... ) >>> df.rows() [('a', 1, 0), ('b', 2, 3), ('b', 3, 6), ('a', 4, 9)] >>> df.rows(named=True) [{'x': 'a', 'y': 1, 'z': 0}, {'x': 'b', 'y': 2, 'z': 3}, {'x': 'b', 'y': 3, 'z': 6}, {'x': 'a', 'y': 4, 'z': 9}] )rr#rr row_tuples)rr,dict_zip_rrs rr-r,s^t #'dllE9=9L9L9NO9N#E$w,-9NO O88&&( (PsA/)r,rr'cgrrrrr,rr's r rows_by_keyDataFrame.rows_by_keyS,s #r)r,rcgrrrs rrr],sr)rr'cgrrrs rrrg,s+.r)rcgrrrs rrrq,s%(rc[X5n[U5S:Xa[URUS55OUR U5R 5nU(aUnO4UR Vs/sH owU;dM UPM nnUR U5n[XVR US95n U(a [U 5n U $[[5n U HupXRU 5 M U $s snf)u Returns all data as a dictionary of python-native values keyed by some column. This method is like `rows`, but instead of returning rows in a flat list, rows are grouped by the values in the `key` column(s) and returned as a dictionary. Note that this method should not be used in place of native operations, due to the high cost of materializing all frame data out into a dictionary; it should be used only when you need to move the values out into a Python data structure or other object that cannot operate directly with Polars/Arrow. Parameters ---------- key The column(s) to use as the key for the returned dictionary. If multiple columns are specified, the key will be a tuple of those values, otherwise it will be a string. named Return dictionary rows instead of tuples, mapping column name to row value. include_key Include key values inline with the associated data (by default the key values are omitted as a memory/performance optimisation, as they can be reoconstructed from the key). unique Indicate that the key is unique; this will result in a 1:1 mapping from key to a single associated row. Note that if the key is *not* actually unique the last row with the given key will be returned. Notes ----- If you have `ns`-precision temporal values you should be aware that Python natively only supports up to `μs`-precision; `ns`-precision values will be truncated to microseconds on conversion to Python. If this matters to your use-case you should export to a different format (such as Arrow or NumPy). See Also -------- rows : Materialize all frame data as a list of rows (potentially expensive). iter_rows : Row iterator over frame data (does not materialize all rows). to_dict : Convert DataFrame to a dictionary mapping column name to values. Examples -------- >>> df = pl.DataFrame( ... { ... "w": ["a", "b", "b", "a"], ... "x": ["q", "q", "q", "k"], ... "y": [1.0, 2.5, 3.0, 4.5], ... "z": [9, 8, 7, 6], ... } ... ) Group rows by the given key column(s): >>> df.rows_by_key(key=["w"]) defaultdict(, {'a': [('q', 1.0, 9), ('k', 4.5, 6)], 'b': [('q', 2.5, 8), ('q', 3.0, 7)]}) Return the same row groupings as dictionaries: >>> df.rows_by_key(key=["w"], named=True) defaultdict(, {'a': [{'x': 'q', 'y': 1.0, 'z': 9}, {'x': 'k', 'y': 4.5, 'z': 6}], 'b': [{'x': 'q', 'y': 2.5, 'z': 8}, {'x': 'q', 'y': 3.0, 'z': 7}]}) Return row groupings, assuming keys are unique: >>> df.rows_by_key(key=["z"], unique=True) {9: ('a', 'q', 1.0), 8: ('b', 'q', 2.5), 7: ('b', 'q', 3.0), 6: ('a', 'k', 4.5)} Return row groupings as dictionaries, assuming keys are unique: >>> df.rows_by_key(key=["z"], named=True, unique=True) {9: {'w': 'a', 'x': 'q', 'y': 1.0}, 8: {'w': 'b', 'x': 'q', 'y': 2.5}, 7: {'w': 'b', 'x': 'q', 'y': 3.0}, 6: {'w': 'a', 'x': 'k', 'y': 4.5}} Return dictionary rows grouped by a compound key, including key values: >>> df.rows_by_key(key=["w", "x"], named=True, include_key=True) defaultdict(, {('a', 'q'): [{'w': 'a', 'x': 'q', 'y': 1.0, 'z': 9}], ('b', 'q'): [{'w': 'b', 'x': 'q', 'y': 2.5, 'z': 8}, {'w': 'b', 'x': 'q', 'y': 3.0, 'z': 7}], ('a', 'k'): [{'w': 'a', 'x': 'k', 'y': 4.5, 'z': 6}]}) rrr+) rfriterrrJ iter_rowsrr#rrrr) rrr,rr'keysrrV data_colszippedr-rs rrr{,sJ *3x1} Q( )S!++- F$(KK@KqC<KI@[[+FT++%+89 >> df = pl.DataFrame( ... { ... "a": [1, 3, 5], ... "b": [2, 4, 6], ... } ... ) >>> [row[0] for row in df.iter_rows()] [1, 3, 5] >>> [row["b"] for row in df.iter_rows(named=True)] [2, 4, 6] rFr+N) rrrr#rFrrYrrar-) rr,rrget_rowrr has_objectrbzerocopy_slicerres rrr-sn)- dhhc(I%%t{{*  z4;; <!%F!@-222?#D$677 @ .222??? =4;;'D'!*566(4;;'aj ( @sB3D"5D 6A+D"c#h# URR5Hn[U5v M g7f)uD Returns an iterator over the columns of this DataFrame. Yields ------ Series Notes ----- Consider whether you can use :func:`all` instead. If you can, it will be more efficient. Examples -------- >>> df = pl.DataFrame( ... { ... "a": [1, 3, 5], ... "b": [2, 4, 6], ... } ... ) >>> [s.name for s in df.iter_columns()] ['a', 'b'] If you're using this to modify a dataframe's columns, e.g. >>> # Do NOT do this >>> pl.DataFrame(column * 2 for column in df.iter_columns()) shape: (3, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 2 ┆ 4 │ │ 6 ┆ 8 │ │ 10 ┆ 12 │ └─────┴─────┘ then consider whether you can use :func:`all` instead: >>> df.select(pl.all() * 2) shape: (3, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 2 ┆ 4 │ │ 6 ┆ 8 │ │ 10 ┆ 12 │ └─────┴─────┘ Nrrs rrDataFrame.iter_columnsP-s(j%%'A)O(s02c#n# [SURU5HnURX!5v M g7f)a Returns a non-copying iterator of slices over the underlying DataFrame. Parameters ---------- n_rows Determines the number of rows contained in each DataFrame slice. Examples -------- >>> from datetime import date >>> df = pl.DataFrame( ... data={ ... "a": range(17_500), ... "b": date(2023, 1, 1), ... "c": "klmnoopqrstuvwxyz", ... }, ... schema_overrides={"a": pl.Int32}, ... ) >>> for idx, frame in enumerate(df.iter_slices()): ... print(f"{type(frame).__name__}:[{idx}]:{len(frame)}") DataFrame:[0]:10000 DataFrame:[1]:7500 Using `iter_slices` is an efficient way to chunk-iterate over DataFrames and any supported frame export/conversion types; for example, as RecordBatches: >>> for frame in df.iter_slices(n_rows=15_000): ... record_batch = frame.to_arrow().to_batches()[0] ... print(f"{record_batch.schema}\n<< {len(record_batch)}") a: int32 b: date32[day] c: large_string << 15000 a: int32 b: date32[day] c: large_string << 2500 See Also -------- iter_rows : Row iterator over frame data (does not materialise all rows). partition_by : Split into multiple DataFrames, partitioned by groups. rN)rYrra)rrrbs r iter_slicesDataFrame.iter_slices-s.ZAt{{F3F**V, ,4s35cU(aURR5 U$UR5nURR5 U$)z\ Shrink DataFrame memory usage. Shrinks to fit the exact capacity needed to hold the data. )r shrink_to_fitr)rrrs rrDataFrame.shrink_to_fit-s<  HH " " $KB FF "IrcjUR[R"S5RX55$)u Take every nth row in the DataFrame and return as a new DataFrame. Parameters ---------- n Gather every *n*-th row. offset Starting index. Examples -------- >>> s = pl.DataFrame({"a": [1, 2, 3, 4], "b": [5, 6, 7, 8]}) >>> s.gather_every(2) shape: (2, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 5 │ │ 3 ┆ 7 │ └─────┴─────┘ >>> s.gather_every(2, offset=1) shape: (2, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 2 ┆ 6 │ │ 4 ┆ 8 │ └─────┴─────┘ r)rJrKra gather_every)rrQrbs rrDataFrame.gather_every-s'H{{155:221=>>rczUnUbUOUnUbUOUnUbUOUn[URRXVXx55$)aT Hash and combine the rows in this DataFrame. The hash value is of type `UInt64`. Parameters ---------- seed Random seed parameter. Defaults to 0. seed_1 Random seed parameter. Defaults to `seed` if not set. seed_2 Random seed parameter. Defaults to `seed` if not set. seed_3 Random seed parameter. Defaults to `seed` if not set. Notes ----- This implementation of `hash_rows` does not guarantee stable results across different Polars versions. Its stability is only guaranteed within a single version. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, None, 3, 4], ... "ham": ["a", "b", None, "d"], ... } ... ) >>> df.hash_rows(seed=42) # doctest: +IGNORE_RESULT shape: (4,) Series: '' [u64] [ 10783150408545073287 1438741209321515184 10047419486152048166 2047317070637311557 ] )r9r hash_rows) rrseed_1seed_2seed_3k0k1k2k3s rrDataFrame.hash_rows-sJ^)Vt)Vt)Vtdhh((899rchUR[R"S5R55$)u: Interpolate intermediate values. The interpolation method is linear. Nulls at the beginning and end of the series remain null. Examples -------- >>> df = pl.DataFrame( ... { ... "foo": [1, None, 9, 10], ... "bar": [6, 7, 9, None], ... "baz": [1, None, None, 9], ... } ... ) >>> df.interpolate() shape: (4, 3) ┌──────┬──────┬──────────┐ │ foo ┆ bar ┆ baz │ │ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ f64 │ ╞══════╪══════╪══════════╡ │ 1.0 ┆ 6.0 ┆ 1.0 │ │ 5.0 ┆ 7.0 ┆ 3.666667 │ │ 9.0 ┆ 9.0 ┆ 6.333333 │ │ 10.0 ┆ null ┆ 9.0 │ └──────┴──────┴──────────┘ r)rJrKra interpolater s rrDataFrame.interpolate!.s$8{{155:11344rc6URR5$)z Returns `True` if the DataFrame contains no rows. Examples -------- >>> df = pl.DataFrame({"foo": [1, 2, 3], "bar": [4, 5, 6]}) >>> df.is_empty() False >>> df.filter(pl.col("foo") > 99).is_empty() True )rr5r s rr5DataFrame.is_empty?.sxx  ""rcL[URRU/55$)a Convert a `DataFrame` to a `Series` of type `Struct`. Parameters ---------- name Name for the struct Series Examples -------- >>> df = pl.DataFrame( ... { ... "a": [1, 2, 3, 4, 5], ... "b": ["one", "two", "three", "four", "five"], ... } ... ) >>> df.to_struct("nums") shape: (5,) Series: 'nums' [struct[2]] [ {1,"one"} {2,"two"} {3,"three"} {4,"four"} {5,"five"} ] )r9r to_structrs rrDataFrame.to_structM.s 8dhh((r233rcSSKJn UR5R"U/UQ76R UR 5S9$)u? Decompose struct columns into separate columns for each of their fields. The new columns will be inserted into the dataframe at the location of the struct column. Parameters ---------- columns Name of the struct column(s) that should be unnested. *more_columns Additional columns to unnest, specified as positional arguments. Examples -------- >>> df = pl.DataFrame( ... { ... "before": ["foo", "bar"], ... "t_a": [1, 2], ... "t_b": ["a", "b"], ... "t_c": [True, None], ... "t_d": [[1, 2], [3]], ... "after": ["baz", "womp"], ... } ... ).select("before", pl.struct(pl.col("^t_.$")).alias("t_struct"), "after") >>> df shape: (2, 3) ┌────────┬─────────────────────┬───────┐ │ before ┆ t_struct ┆ after │ │ --- ┆ --- ┆ --- │ │ str ┆ struct[4] ┆ str │ ╞════════╪═════════════════════╪═══════╡ │ foo ┆ {1,"a",true,[1, 2]} ┆ baz │ │ bar ┆ {2,"b",null,[3]} ┆ womp │ └────────┴─────────────────────┴───────┘ >>> df.unnest("t_struct") shape: (2, 6) ┌────────┬─────┬─────┬──────┬───────────┬───────┐ │ before ┆ t_a ┆ t_b ┆ t_c ┆ t_d ┆ after │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ str ┆ bool ┆ list[i64] ┆ str │ ╞════════╪═════╪═════╪══════╪═══════════╪═══════╡ │ foo ┆ 1 ┆ a ┆ true ┆ [1, 2] ┆ baz │ │ bar ┆ 2 ┆ b ┆ null ┆ [3] ┆ womp │ └────────┴─────┴─────┴──────┴───────────┴───────┘ rrr)rrrr7rrrs rr7DataFrame.unnestk.sGf = IIK V  ,* , W=#7#7#9W : rc [R"UR54SS0UD6nURS:Xa[R"U/5n[ X R S9$)uY Return pairwise Pearson product-moment correlation coefficients between columns. See numpy `corrcoef` for more information: https://numpy.org/doc/stable/reference/generated/numpy.corrcoef.html Notes ----- This functionality requires numpy to be installed. Parameters ---------- **kwargs Keyword arguments are passed to numpy `corrcoef`. Examples -------- >>> df = pl.DataFrame({"foo": [1, 2, 3], "bar": [3, 2, 1], "ham": [7, 8, 9]}) >>> df.corr() shape: (3, 3) ┌──────┬──────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ f64 │ ╞══════╪══════╪══════╡ │ 1.0 ┆ -1.0 ┆ 1.0 │ │ -1.0 ┆ 1.0 ┆ -1.0 │ │ 1.0 ┆ -1.0 ┆ 1.0 │ └──────┴──────┴──────┘ rowvarFrr)rcorrcoefr*rrrr)rrcorrelation_matrixs rcorrDataFrame.corr.sQ> [[QQ&Q ::?!#+=*>!? +LLAArcSSKJn [X5 UR5R UR5U5R UR 5S9$)u4 Take two sorted DataFrames and merge them by the sorted key. The output of this operation will also be sorted. It is the callers responsibility that the frames are sorted in ascending order by that key otherwise the output will not make sense. The schemas of both DataFrames must be equal. Parameters ---------- other Other DataFrame that must be merged key Key that is sorted. Examples -------- >>> df0 = pl.DataFrame( ... {"name": ["steve", "elise", "bob"], "age": [42, 44, 18]} ... ).sort("age") >>> df0 shape: (3, 2) ┌───────┬─────┐ │ name ┆ age │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═══════╪═════╡ │ bob ┆ 18 │ │ steve ┆ 42 │ │ elise ┆ 44 │ └───────┴─────┘ >>> df1 = pl.DataFrame( ... {"name": ["anna", "megan", "steve", "thomas"], "age": [21, 33, 42, 20]} ... ).sort("age") >>> df1 shape: (4, 2) ┌────────┬─────┐ │ name ┆ age │ │ --- ┆ --- │ │ str ┆ i64 │ ╞════════╪═════╡ │ thomas ┆ 20 │ │ anna ┆ 21 │ │ megan ┆ 33 │ │ steve ┆ 42 │ └────────┴─────┘ >>> df0.merge_sorted(df1, key="age") shape: (7, 2) ┌────────┬─────┐ │ name ┆ age │ │ --- ┆ --- │ │ str ┆ i64 │ ╞════════╪═════╡ │ bob ┆ 18 │ │ thomas ┆ 20 │ │ anna ┆ 21 │ │ megan ┆ 33 │ │ steve ┆ 42 │ │ steve ┆ 42 │ │ elise ┆ 44 │ └────────┴─────┘ Notes ----- No guarantee is given over the output row order when the key is equal between the both dataframes. The key must be sorted in ascending order. rrr)rrr4r merge_sortedrr)rr:rrs rrDataFrame.merge_sorted.sGP =$& IIK \%**, , W=#7#7#9W : rr:c~SSKJn UR5RXS9R UR 5S9$)a Flag a column as sorted. This can speed up future operations. Parameters ---------- column Column that is sorted descending Whether the column is sorted in descending order. Warnings -------- This can lead to incorrect results if the data is NOT sorted!! Use with care! rrrr)rrr set_sortedrr)rrr:rs rrDataFrame.set_sorted/s84 = IIK ZZ 6 W=#7#7#9W : rrr include_nullsr=c SSKJn [X5 UR5R UR5UUUUUUS9R UR 5S9$)u  Update the values in this `DataFrame` with the values in `other`. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. Parameters ---------- other DataFrame that will be used to update the values on Column names that will be joined on. If set to `None` (default), the implicit row index of each frame is used as a join key. how : {'left', 'inner', 'full'} * 'left' will keep all rows from the left table; rows may be duplicated if multiple rows in the right frame match the left row's key. * 'inner' keeps only those rows where the key exists in both frames. * 'full' will update existing rows where the key matches while also adding any new rows contained in the given frame. left_on Join column(s) of the left DataFrame. right_on Join column(s) of the right DataFrame. include_nulls Overwrite values in the left frame with null values from the right frame. If set to `False` (default), null values in the right frame are ignored. maintain_order : {'none', 'left', 'right', 'left_right', 'right_left'} Which order of rows from the inputs to preserve. See :func:`~DataFrame.join` for details. Unlike `join` this function preserves the left order by default. Notes ----- This is syntactic sugar for a left/inner join that preserves the order of the left `DataFrame` by default, with an optional coalesce when `include_nulls = False`. Examples -------- >>> df = pl.DataFrame( ... { ... "A": [1, 2, 3, 4], ... "B": [400, 500, 600, 700], ... } ... ) >>> df shape: (4, 2) ┌─────┬─────┐ │ A ┆ B │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 400 │ │ 2 ┆ 500 │ │ 3 ┆ 600 │ │ 4 ┆ 700 │ └─────┴─────┘ >>> new_df = pl.DataFrame( ... { ... "B": [-66, None, -99], ... "C": [5, 3, 1], ... } ... ) Update `df` values with the non-null values in `new_df`, by row index: >>> df.update(new_df) shape: (4, 2) ┌─────┬─────┐ │ A ┆ B │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ -66 │ │ 2 ┆ 500 │ │ 3 ┆ -99 │ │ 4 ┆ 700 │ └─────┴─────┘ Update `df` values with the non-null values in `new_df`, by row index, but only keeping those rows that are common to both frames: >>> df.update(new_df, how="inner") shape: (3, 2) ┌─────┬─────┐ │ A ┆ B │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ -66 │ │ 2 ┆ 500 │ │ 3 ┆ -99 │ └─────┴─────┘ Update `df` values with the non-null values in `new_df`, using a full outer join strategy that defines explicit join columns in each frame: >>> df.update(new_df, left_on=["A"], right_on=["C"], how="full") shape: (5, 2) ┌─────┬─────┐ │ A ┆ B │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ -99 │ │ 2 ┆ 500 │ │ 3 ┆ 600 │ │ 4 ┆ 700 │ │ 5 ┆ -66 │ └─────┴─────┘ Update `df` values including null values in `new_df`, using a full outer join strategy that defines explicit join columns in each frame: >>> df.update(new_df, left_on="A", right_on="C", how="full", include_nulls=True) shape: (5, 2) ┌─────┬──────┐ │ A ┆ B │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪══════╡ │ 1 ┆ -99 │ │ 2 ┆ 500 │ │ 3 ┆ null │ │ 4 ┆ 700 │ │ 5 ┆ -66 │ └─────┴──────┘ rrrr)rrr4rupdaterr) rr:rrBrrrr=rs rrDataFrame.update>/s_Z =$& IIK V !+-W=#7#7#9W : rcSSKJn UR5R5R UR 5S9$)u Return the number of non-null elements for each column. Examples -------- >>> df = pl.DataFrame( ... {"a": [1, 2, 3, 4], "b": [1, 2, 1, None], "c": [None, None, None, None]} ... ) >>> df.count() shape: (1, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ u32 ┆ u32 ┆ u32 │ ╞═════╪═════╪═════╡ │ 4 ┆ 3 ┆ 0 │ └─────┴─────┴─────┘ rrr)rrrrrrrLs rrDataFrame.count/s3& =yy{  "**9M9M9O*PPrz`DataFrame.melt` is deprecated; use `DataFrame.unpivot` instead, with `index` instead of `id_vars` and `on` instead of `value_vars`c&URUUUUS9$)a@ Unpivot a DataFrame from wide to long format. Optionally leaves identifiers set. This function is useful to massage a DataFrame into a format where one or more columns are identifier variables (id_vars) while all other columns, considered measured variables (value_vars), are "unpivoted" to the row axis leaving just two non-identifier columns, 'variable' and 'value'. .. deprecated:: 1.0.0 Use the :meth:`.unpivot` method instead. Parameters ---------- id_vars Column(s) or selector(s) to use as identifier variables. value_vars Column(s) or selector(s) to use as values variables; if `value_vars` is empty all columns that are not in `id_vars` will be used. variable_name Name to give to the `variable` column. Defaults to "variable" value_name Name to give to the `value` column. Defaults to "value" )rrrr)r )rid_vars value_varsrrs rmeltDataFrame.melt/s'H||'!   rraiseforbid)missing_columnsmissing_struct_fields extra_columnsextra_struct_fields integer_cast float_castc SSKJn UR5RUUUUUUUS9R UR 5S9$)u Match or evolve the schema of a LazyFrame into a specific schema. By default, match_to_schema returns an error if the input schema does not exactly match the target schema. It also allows columns to be freely reordered, with additional coercion rules available through optional parameters. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. Parameters ---------- schema Target schema to match or evolve to. missing_columns Raise of insert missing columns from the input with respect to the `schema`. This can also be an expression per column with what to insert if it is missing. missing_struct_fields Raise of insert missing struct fields from the input with respect to the `schema`. extra_columns Raise of ignore extra columns from the input with respect to the `schema`. extra_struct_fields Raise of ignore extra struct fields from the input with respect to the `schema`. integer_cast Forbid of upcast for integer columns from the input to the respective column in `schema`. float_cast Forbid of upcast for float columns from the input to the respective column in `schema`. Examples -------- Ensuring the schema matches >>> df = pl.DataFrame({"a": [1, 2, 3], "b": ["A", "B", "C"]}) >>> df.match_to_schema({"a": pl.Int64, "b": pl.String}) shape: (3, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ str │ ╞═════╪═════╡ │ 1 ┆ A │ │ 2 ┆ B │ │ 3 ┆ C │ └─────┴─────┘ >>> df.match_to_schema({"a": pl.Int64}) # doctest: +SKIP polars.exceptions.SchemaError: extra columns in `match_to_schema`: "b" Adding missing columns >>> ( ... pl.DataFrame({"a": [1, 2, 3]}).match_to_schema( ... {"a": pl.Int64, "b": pl.String}, ... missing_columns="insert", ... ) ... ) shape: (3, 2) ┌─────┬──────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ str │ ╞═════╪══════╡ │ 1 ┆ null │ │ 2 ┆ null │ │ 3 ┆ null │ └─────┴──────┘ >>> ( ... pl.DataFrame({"a": [1, 2, 3]}).match_to_schema( ... {"a": pl.Int64, "b": pl.String}, ... missing_columns={"b": pl.col.a.cast(pl.String)}, ... ) ... ) shape: (3, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ str │ ╞═════╪═════╡ │ 1 ┆ 1 │ │ 2 ┆ 2 │ │ 3 ┆ 3 │ └─────┴─────┘ Removing extra columns >>> ( ... pl.DataFrame({"a": [1, 2, 3], "b": ["A", "B", "C"]}).match_to_schema( ... {"a": pl.Int64}, ... extra_columns="ignore", ... ) ... ) shape: (3, 1) ┌─────┐ │ a │ │ --- │ │ i64 │ ╞═════╡ │ 1 │ │ 2 │ │ 3 │ └─────┘ Upcasting integers and floats >>> ( ... pl.DataFrame( ... {"a": [1, 2, 3], "b": [1.0, 2.0, 3.0]}, ... schema={"a": pl.Int32, "b": pl.Float32}, ... ).match_to_schema( ... {"a": pl.Int64, "b": pl.Float64}, ... integer_cast="upcast", ... float_cast="upcast", ... ) ... ) shape: (3, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ f64 │ ╞═════╪═════╡ │ 1 ┆ 1.0 │ │ 2 ┆ 2.0 │ │ 3 ┆ 3.0 │ └─────┴─────┘ rr)rrrrrrrr)rrrmatch_to_schemarr) rrrrrrrrrs rrDataFrame.match_to_schema0sSh = IIK _ /&;+$7)%W=#7#7#9W : rc$UnUb)[U[5(aU/nURU5nURURR 55nUb5[U[5(aU/nSU;aS/U-nURU5nU$)z Get all runtime metadata for each column. This is unstable and is meant for debugging purposes. Parameters ---------- columns Column(s) to show the information for stats Statistics to show column_name)rrrJrr _to_metadata)rrstatsrmds rrDataFrame._to_metadata0s"  '3''")7#B __RVV002 3  %%%E)&%/5!B r unorderedr:r;c UR[R"[R"5UUUS95R 5$)z Row encode the given DataFrame. This is an internal function not meant for outside consumption and can be changed or removed at any point in time. fields have order: - descending - nulls_last - no_order r )r>rK _row_encoderLr )rr r:r;s rr DataFrame._row_encode0s<$ MM#%%    )+ r)r)NN)rzFrameInitTypes | NonerSchemaDefinition | NonerSchemaDict | Nonerr[rzOrientation | Noner int | Nonerr[rNone)rzstr | Path | IOBaserrrr)rrgrrr) rzpa.Table | pa.RecordBatchrrrrrr[rr)r pd.DataFramerrrrrr[rr[rr[rr)rrrrurr)rrrrrr)rr>)rro)rztuple[int, int])rr)rz list[str])rz Sequence[str]rr)rzlist[DataType])rzdict[str, dict[str, bool]])rrd)r+znpt.DTypeLike | Noner. bool | Nonernp.ndarray[Any, Any]r&)r4r[r)r[rr)r:rr;r}rr)r:rr;r}rr)r:rrcr[rr)rrrlzfrozenset[PolarsDataType]rmrrr)r:z DataFrame | Series | int | floatrr)rr)r:objectrr)r:rrr)rbytes)rrrr)r: int | floatrr)r:z-DataFrame | Series | int | float | bool | strrrr)rrrr[)rzIterator[Series])rz-tuple[SingleIndexSelector, SingleColSelector]rr)rz2str | tuple[MultiIndexSelector, SingleColSelector]rru)rzSingleIndexSelector | MultiIndexSelector | MultiColSelector | tuple[SingleIndexSelector, MultiColSelector] | tuple[MultiIndexSelector, MultiColSelector]rr)ra SingleIndexSelector | SingleColSelector | MultiColSelector | MultiIndexSelector | tuple[SingleIndexSelector, SingleColSelector] | tuple[SingleIndexSelector, MultiColSelector] | tuple[MultiIndexSelector, SingleColSelector] | tuple[MultiIndexSelector, MultiColSelector]rzDataFrame | Series | Any)rz;str | Sequence[int] | Sequence[str] | tuple[Any, str | int]rrrr)rr)rrrr)rz object | Nonerr)rr[rr)rrrzint | str | Nonerr)rCompatLevel | Nonerzpa.Table)r  Literal[True]rzdict[str, Series])r Literal[False]rzdict[str, list[Any]])r r[rz(dict[str, Series] | dict[str, list[Any]])rlist[dict[str, Any]]) r1rr(r[r)r[r2r[r3rrr).)rFzLiteral['array']rAjax.Device | str | NonerB(str | Expr | Sequence[str | Expr] | NonerCrr+PolarsDataType | Noner1rrz jax.Array)rFLiteral['dict']rArrBrrCrr+rr1rrzdict[str, jax.Array])r)rFrrArrBrrCrr+rr1rrz jax.Array | dict[str, jax.Array]) rFzLiteral['tensor']rBrrCrr+rrz torch.Tensor) rFzLiteral['dataset']rBrrCrr+rrr) rFr rBrrCrr+rrzdict[str, torch.Tensor])rr) rFrrBrrCrr+rrz6torch.Tensor | dict[str, torch.Tensor] | PolarsDataset)ryr[rrrr)rrryr[rrrr)r)rrrru)i)rQrrr)rrrzLiteral['binary']rr)rrrzLiteral['json']rr)rIOBase | str | Pathrrrr)rIOBase | str | Path | Nonerrrzbytes | str | None)rrrr)rr!rr)rr"r str | None)rz str | Path | IO[bytes] | IO[str]rr)rz'str | Path | IO[bytes] | IO[str] | Nonerr#)&rrrr[rr[rrrrrrrrrr#rr#rr#rrrrrr[rr#rCsvQuoteStyle | Nonerdict[str, Any] | Noner3CredentialProviderFunction | Literal['auto'] | Nonerrrr)&rz str | Path | IO[str] | IO[bytes]rr[rr[rrrrrrrrrr#rr#rr#rrrrrr[rr#rr$rr%rr&rrrr)&rz'str | Path | IO[str] | IO[bytes] | Nonerr[rr[rrrrrrrrrr#rr#rr#rrrrrr[rr#rr$rr%rr&rrrr#)rrrrrr)rr)rstr | Path | IO[bytes]rrwrrrr)0r.z(str | Workbook | IO[bytes] | Path | Noner/zstr | Worksheet | Nonerztuple[int, int] | strrzstr | dict[str, Any] | Nonerr#rzColumnFormatDict | Nonerz$dict[OneOrMoreDataTypes, str] | NonerzConditionalFormatDict | Nonerr%rzColumnTotalsDefinition | NonerzColumnWidthsDefinition | NonerzRowTotalsDefinition | Nonerz-dict[int | tuple[int, ...], int] | int | Nonerz0dict[str, Sequence[str] | dict[str, Any]] | Nonerz&dict[str, str | dict[str, str]] | Nonerrrr[rr[rr[r z#Sequence[str] | SelectorType | Noner r[r rr zOstr | tuple[int, int] | tuple[str, int, int] | tuple[int, int, int, int] | Nonerrp)rrrrrrrr%rr&rrrr )rr'rrrrrr%rr&rrrr)rstr | Path | IO[bytes] | Nonerrrrrr%rr&rrrBytesIO | None)rrrrrrrr )rr'rrrrrr)rr(rrrrrr)) rr'rrrNrrOzbool | str | dict[str, bool]rPrrQrr3r[rRr%rSstr | Sequence[str] | NonerTrrr%rr&rrrUzParquetMetadata | NonerVr[rr) rrrzConnectionOrCursor | strrlrrzDbWriteEngine | Nonermr%rr)rRzstr | pyiceberg.table.TablerzLiteral['append', 'overwrite']rr)rR!str | Path | deltalake.DeltaTablerz1Literal['error', 'append', 'overwrite', 'ignore']rrrdict[str, str] | Nonerr&rr%rr)rRr+rzLiteral['merge']rrrr,rr&rdict[str, Any]rzdeltalake.table.TableMerger)rRr+rz:Literal['error', 'append', 'overwrite', 'ignore', 'merge']rrrr,rr&rr%rr%rz"deltalake.table.TableMerger | None)b)rrrr)rr[rrrzstr | Iterable[str] | Nonerr)rz(Mapping[str, str] | Callable[[str], str]rr[rr)rrrrrr)rzTIntoExprColumn | Iterable[IntoExprColumn] | bool | list[bool] | np.ndarray[Any, Any]rrrr)r rr rr rrr)r rr rr rrr)r rr rr r[rr#))g?g?g?)r2zSequence[float] | float | Noner0rrr)rrrr)rrrrurr)r`IntoExpr | Iterable[IntoExpr]rArr:bool | Sequence[bool]r;r0r<r[r=r[rr)rIrrrrr)rVrr`r/rr0rr)r:rr\r[rr[)rbrrWrrr)r)rQrrr)rr>ColumnNameOrSelector | Collection[ColumnNameOrSelector] | Nonerr)rxz&Callable[Concatenate[DataFrame, P], T]ryP.argsrP.kwargsrr) rz!str | Sequence[str] | pl.SelectorrxzCallable[[Series], Series]ryr2rr3rr)rr)rrrbrrr)row_nrr)r`r/r=r[rrrr<) rrrstr | timedeltarbstr | timedelta | Nonerrxr$IntoExpr | Iterable[IntoExpr] | Nonerr=)rrrr5rr6rbr6rr[rrxrBrrr7rrrr;) rrrr5rr*r=r[rr) r:rrstr | None | Exprrr8rr8rr*rr*r`r*rrvrMrrz$str | int | float | timedelta | Nonerr[rr[rr[rr[rr[rr)Ninner)r:rrrrBrrrrrrMrrrrr[rrr=MaintainOrderJoin | Nonerr)r:rrzExpr | Iterable[Expr]rMrrr)rxz Callable[[tuple[Any, ...]], Any]rrrrrr)rzlist[Series] | DataFramerr[rr)r:rrr[rr)r:rrr)r5ColumnNameOrSelector | Iterable[ColumnNameOrSelector]rr[rr)rrrru)rz`Mapping[ColumnNameOrSelector | PolarsDataType, PolarsDataType | PythonDataType] | PolarsDataTyperr[rr)rz list[Series])rrrzSeries | NoDefaultrru)rrrrrr)rrrzAny | NoDefaultrz Series | Any)NNN) rzAny | Expr | NonerzFillNullStrategy | Nonermrrr[rr)rzExpr | int | float | Nonerr)rr;rrzrr)r5ColumnNameOrSelector | Sequence[ColumnNameOrSelector]rr]rCrGr+rPr*rVrr]rrcrfrlrrrruryr'rrrrrrrr-rrrrrrrrr5rr7rrrrrrrrr __static_attributes__rrrrrs(Xt &,g%6J"6'+*.e! /3%)*9!e!#e!(e! , e!  e!#e!(e!e! e!NKS94(945H94 9494v +/) /3 ) ') () , )  )  ) ) V+/. /3 #. . (. , .  . . .  . . ` KK Z]#]#~ Z3%3%j    !!   *$"$"L ^^ ) )&!&!P A AJJ&GK ) 8C   H"0<0<0<  077$=7CQ7 7 /0'(''** 877B7 7B 7 7 !##, @ E   :    I- :I- "I-Va! Ha!a!  a!F==38  6)V1'1'f!>5I=A)5J)5V47RR PP 7 7 177 $(]7 ]7 1]7~%,& #'WRWR WR  WR  WR!WR WRr), +.:==@'* % (  8  ; %      +.:==@'* #$ #( # 8 # ; #% # #  # #Z&-u&+/:>=A'+%u&"u&( u& 8 u& ; u&%u&u& *u&u&n*-;>=@'* &8  ;  %   ;>=@'* '8  ;  %   ;>=@'* &$&8 & ; & % & !&&Z(0K";?=A'+ K"$K"8 K" ; K" % K" @K"K"`-2` &*`  `  ` D#&*# #  #JF F&* F  F  F.11B7!7!r?B+< QQ JM'4G  ,0>A'/ >A(>A$ >A  >A@66 @@,,\99 OO?C.;. .` ""&)"%"%(+&)!!$,/14SV)     $  &$ !"*#$/%&Q'()* +.  ""&)"%"%(+&)!!$,/14SV).     $  &$ !"*#$/%&Q'()* +29=l"##&*"&"&(,&*#!%,015 -l5l l  l  llll$l l l&l$ll !l"*#l$/%l( @)l,-l. /l\37((0(6 '5$'5%'5 '5  '5V>B,0n +/37!%26>B<@/37;7;15EIGK;? #>B$!% ?n:n*n ( n 1 nn0n<n:n-n5n5n/nCn E!n"9#n$%n&'n()n*+n,<-n./n01n4 5n@ An` '5+/15   $  )  /  @      '5+/15  $ $  )  /  @     !>5I '5+/15 X0+X0$ X0 ) X0 / X0 @X0X0 X0JX0t '5+/ $  )    '5+/ $$  )   !>5I '5+/ 8.+8.$ 8. ) 8.  8.J8.|+1(,37%)%)!1537*715 +/'l $l ( l & l 1 l #l #l l /l 1l %(l /l  @l "#l $)%l &'l ( )l f(.'+04K!K!-K! % K! % K!.K! K!ZZ$"+$"-$"  $"$"L CF(+14SV58 1 @  &  / Q 3     ),14SV *1 * * & * / *Q *, * % * *LS(,155959A1AI A & A / AA3A3A ,AF+%+%` %#37 xQxQ xQ 1 xQ  xQt16TX2 ?2 LP2 2 hFPl  #l l  l \O  #O O  O b%("%+. "  )   %("% "  (  %("% "     %'"$!& U"U U  U  Ur7Ic )2 c 3c & c  c J/*&X-2,1"$l )l l * l * l  l l  l \4:Z&Z&x!y'J */ V V  * V ' V  V KV p!y'J */ V V  * V ' V  V KV p>B%A%AN"?"?H2121h2121h''VRVX NX  X xRVw Nw  w rA/8A/A/ A/ A/F[ 7[ -[  [  [  [ zF,F,P S#1 #1P %SM *SMSM SM  SMj!z9E *.!(9=Z Z  Z ' Z  Z 7Z  Z FZ x!z9E *.)-#(!'9=$} }  } ' } ' } !} } } 7} }  } F} ~ !z9E 04$ c c  c - c  c  c Fc R&*&* $.2/3)-%/:>#$$(!%#_ _ # _ $ _  _ ,_ -_  '_ #_ _ 8_ _ _ _ "!_ "#_ $ %_ B !}fM8<# t =A=A#(! $37t t  5t  t : t ;t t !t t t 1t  t Nt lZ  e e +e  e  e e T/3d* " d*2d*,d*  d*  d*NFK)M/)M>B)M )MVE -E  E   E N& -&  &   & PDDD BB""""HO6F8O6F8O66:""HP87;""H)S)SV)S)SVR6"D@I& & .<& & TQUQ   Q MQ  Q  Q  Q  Q Q jRV[ $)$ [ N[ ! [  [  [ [ zJ4J4X <  B366"&FY+/!&FY FY( FY  FY  FYFY FYFYP[z %( # "      %( "     !h"%) h"h"" h"  h" * h"h"T.1PPP HH %?)?) 5?)B !$!$ # B# #  #  # ## !$  B        !$ . B. .  .  . )..  ( B( (  (  ( #((! ~ B~ ~  ~  ~ ~@),(&(;>( "((:='%'47' !'' %I!I!36I! =I!V6p.-.-`16  $?$?P!!! 3:3:3: 3:  3:  3:j5< #44<9 H9 ,9   9 v"BHP l!        DZ*.06 [ /3/3#39[ [  '[ . [ , [ -[ [ 1[  [ [ zQ. H QUSW$(!% % M% Q% " %  %  % % NZ =D5<4;5<6>6>a #a : a  3 a 2a 3a 4a 4a  a a J+/(,$'$&$  $R (,(, &  &   rrcRUn[U[R5(dP[U[5(aO"[U[5(a Sn[ U5e[R"SU/5nUb2US:aUR X!S- S9nU$US:XaURSS5nU$)Nzoperation not supportedrr)rrQr)rrrurrrrra)r:rWrrs rrXrX1s E eRYY ' ' eS ! !  x ( (+CC.  "ug&  A:))!)DE Lq[KK1%E Lrr)r:rrWrrru)rF __future__rr]rr collectionsrcollections.abcrrrrr ior r pathlibr typingr rrrrrrrrrpolars._reexport _reexportrpolarsrrKpolars._typingrrrpolars._utils.constructionrrrrrr r!r"polars._utils.convertr#polars._utils.deprecationr$r%r&polars._utils.getitemr(polars._utils.parser)polars._utils.pycapsuler*r+polars._utils.serder,polars._utils.unstabler-r.polars._utils.variousr/r0r1r2r3r4r5r6polars._utils.wrapr7r8r9polars.dataframe._htmlr:polars.dataframe.group_byr;r<r=polars.dataframe.plottingr>polars.datatypesr?r@rArBrCrDrErFrGrHrIrJrKpolars.datatypes.grouprLpolars.dependenciesrMrNrOrPrQrRrSrTrUrVrWrXrYrrZrr[rpolars.exceptionsr\r]r^r_r`polars.functionsrarbpolars.interchange.protocolrc polars.schemardrrerfsuppress ImportError polars._plrrgrhrrirsysrjrkdatetimerlrmrnrrL numpy.typingnpt pyicebergrorrpxlsxwriter.worksheetrqrrrsrtrurvrwrxryrzr{r|r}r~rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr3rpolars.io.cloudrrtr version_inforrtyping_extensionswarningsrrrrXrrrrxs:" #!   !FF   ; 55D7C   ;:4MM32    ,,-&3 F%'=M& =="#.88............^0<:- 7"11< 7"'0 A#Ax@x@vAeD&%s .J99 K