'h\J S SK Jr S SKrS SKJr S SKJr S SKJrJr S SK J
r
J r S SK J
r
S SKJrJr S S KJr S S
KJr S S KJr S S KJr S S
KJr \( a S SKJr S SKJ r S SK J r J r J r S SK!J"r" SSSSSSSSS. S S jjr#SSSSSSSS. S S jjr$SS.S S jjr% S! S"S jjr&S#S jr'S$S jr(g)% ) annotationsN)datetime)Path)
TYPE_CHECKINGAny)NullTime)
unpack_dtypes)_DELTALAKE_AVAILABLE deltalake)_get_path_scheme) scan_parquet)scan_pyarrow_dataset)ScanCastOptions)Schema)Literal
DeltaTable) DataFrameDataType LazyFrame)CredentialProviderFunctionautoF)versioncolumnsrechunkstorage_optionscredential_providerdelta_table_options use_pyarrowpyarrow_optionsc j [ U UUUUUUUS9n Ub U R U5 n U R 5 $ )a
Reads into a DataFrame from a Delta lake table.
Parameters
----------
source
DeltaTable or a Path or URI to the root of the Delta lake table.
Note: For Local filesystem, absolute and relative paths are supported but
for the supported object storages - GCS, Azure and S3 full URI must be provided.
version
Numerical version or timestamp version of the Delta lake table.
Note: If `version` is not provided, the latest version of delta lake
table is read.
columns
Columns to select. Accepts a list of column names.
rechunk
Make sure that all columns are contiguous in memory by
aggregating the chunks into a single array.
storage_options
Extra options for the storage backends supported by `deltalake`.
For cloud storages, this may include configurations for authentication etc.
More info is available `here
`__.
credential_provider
Provide a function that can be called to provide cloud storage
credentials. The function is expected to return a dictionary of
credential keys along with an optional credential expiry time.
.. warning::
This functionality is considered **unstable**. It may be changed
at any point without it being considered a breaking change.
delta_table_options
Additional keyword arguments while reading a Delta lake Table.
use_pyarrow
Flag to enable pyarrow dataset reads.
pyarrow_options
Keyword arguments while converting a Delta lake Table to pyarrow table.
Returns
-------
DataFrame
Examples
--------
Reads a Delta table from local filesystem.
Note: Since version is not provided, the latest version of the delta table is read.
>>> table_path = "/path/to/delta-table/"
>>> pl.read_delta(table_path) # doctest: +SKIP
Reads a specific version of the Delta table from local filesystem.
Note: This will fail if the provided version of the delta table does not exist.
>>> pl.read_delta(table_path, version=1) # doctest: +SKIP
Time travel a delta table from local filesystem using a timestamp version.
>>> pl.read_delta(
... table_path, version=datetime(2020, 1, 1, tzinfo=timezone.utc)
... ) # doctest: +SKIP
Reads a Delta table from AWS S3.
See a list of supported storage options for S3 `here
`__.
>>> table_path = "s3://bucket/path/to/delta-table/"
>>> storage_options = {
... "AWS_ACCESS_KEY_ID": "THE_AWS_ACCESS_KEY_ID",
... "AWS_SECRET_ACCESS_KEY": "THE_AWS_SECRET_ACCESS_KEY",
... }
>>> pl.read_delta(table_path, storage_options=storage_options) # doctest: +SKIP
Reads a Delta table from Google Cloud storage (GCS).
See a list of supported storage options for GCS `here
`__.
>>> table_path = "gs://bucket/path/to/delta-table/"
>>> storage_options = {"SERVICE_ACCOUNT": "SERVICE_ACCOUNT_JSON_ABSOLUTE_PATH"}
>>> pl.read_delta(table_path, storage_options=storage_options) # doctest: +SKIP
Reads a Delta table from Azure.
Following type of table paths are supported,
* az:///
* adl:///
* abfs:///
See a list of supported storage options for Azure `here
`__.
>>> table_path = "az://container/path/to/delta-table/"
>>> storage_options = {
... "AZURE_STORAGE_ACCOUNT_NAME": "AZURE_STORAGE_ACCOUNT_NAME",
... "AZURE_STORAGE_ACCOUNT_KEY": "AZURE_STORAGE_ACCOUNT_KEY",
... }
>>> pl.read_delta(table_path, storage_options=storage_options) # doctest: +SKIP
Reads a Delta table with additional delta specific options. In the below example,
`without_files` option is used which loads the table without file tracking
information.
>>> table_path = "/path/to/delta-table/"
>>> delta_table_options = {"without_files": True}
>>> pl.read_delta(
... table_path, delta_table_options=delta_table_options
... ) # doctest: +SKIP
)sourcer r r r r r! r )
scan_deltaselectcollect)
r# r r r r r r r r! dfs
bC:\Users\julio\OneDrive\Documentos\Trabajo\Ideas Frescas\venv\Lib\site-packages\polars/io/delta.py
read_deltar) sI v
' / / '
B
YYw
::< )r r r r r r! r c
[ 5 0 nSSKJn SSKJn
SSKJn [ X 5 ( d U
" X0US5 n OUb US:w a
Sn
[ U
5 eSn AU ( a( U R 5 =n( a U " U5 =( d 0 n[ U UUc U b 0 U=( d 0 EUEOSUS 9n[ X 5 ( a2 U R c Ub" 0 U R =( d 0 EU=( d 0 EnU( a( U=( d 0 nUR " S0 UD6n[ U5 $ Ub
S
n
[ U
5 eSS KJn SS KJnJnJn UR) 5 nUR* U: d UR* U:X a S
UR* SU SU 3n
U" U
5 eUR* S: aG UR, b: 1 UR, kR/ U5 n[1 U5 S: a SU S3n
U" U
5 eUR3 5 n[5 U5 nUR7 5 R8 n SS jnU" UU5 u nn UR; 5 n UR< R? S5 ( a" U V s/ s H n U RA SS5 PM n n [C U U[1 U5 S: a U OS[D RF " 5 SS[1 U5 S: UU U=( d SS9
$ s sn f ) a
Lazily read from a Delta lake table.
Parameters
----------
source
DeltaTable or a Path or URI to the root of the Delta lake table.
Note: For Local filesystem, absolute and relative paths are supported but
for the supported object storages - GCS, Azure and S3 full URI must be provided.
version
Numerical version or timestamp version of the Delta lake table.
Note: If `version` is not provided, the latest version of delta lake
table is read.
storage_options
Extra options for the storage backends supported by `deltalake`.
For cloud storages, this may include configurations for authentication etc.
More info is available `here
`__.
credential_provider
Provide a function that can be called to provide cloud storage
credentials. The function is expected to return a dictionary of
credential keys along with an optional credential expiry time.
.. warning::
This functionality is considered **unstable**. It may be changed
at any point without it being considered a breaking change.
delta_table_options
Additional keyword arguments while reading a Delta lake Table.
use_pyarrow
Flag to enable pyarrow dataset reads.
pyarrow_options
Keyword arguments while converting a Delta lake Table to pyarrow table.
Use this parameter when filtering on partitioned columns or to read
from a 'fsspec' supported filesystem.
rechunk
Make sure that all columns are contiguous in memory by
aggregating the chunks into a single array.
Returns
-------
LazyFrame
Examples
--------
Creates a scan for a Delta table from local filesystem.
Note: Since version is not provided, the latest version of the delta table is read.
>>> table_path = "/path/to/delta-table/"
>>> pl.scan_delta(table_path).collect() # doctest: +SKIP
Creates a scan for a specific version of the Delta table from local filesystem.
Note: This will fail if the provided version of the delta table does not exist.
>>> pl.scan_delta(table_path, version=1).collect() # doctest: +SKIP
Time travel a delta table from local filesystem using a timestamp version.
>>> pl.scan_delta(
... table_path, version=datetime(2020, 1, 1, tzinfo=timezone.utc)
... ).collect() # doctest: +SKIP
Creates a scan for a Delta table from AWS S3.
See a list of supported storage options for S3 `here
`__.
>>> table_path = "s3://bucket/path/to/delta-table/"
>>> storage_options = {
... "AWS_REGION": "eu-central-1",
... "AWS_ACCESS_KEY_ID": "THE_AWS_ACCESS_KEY_ID",
... "AWS_SECRET_ACCESS_KEY": "THE_AWS_SECRET_ACCESS_KEY",
... }
>>> pl.scan_delta(
... table_path, storage_options=storage_options
... ).collect() # doctest: +SKIP
Creates a scan for a Delta table from Google Cloud storage (GCS).
See a list of supported storage options for GCS `here
`__.
>>> table_path = "gs://bucket/path/to/delta-table/"
>>> storage_options = {"SERVICE_ACCOUNT": "SERVICE_ACCOUNT_JSON_ABSOLUTE_PATH"}
>>> pl.scan_delta(
... table_path, storage_options=storage_options
... ).collect() # doctest: +SKIP
Creates a scan for a Delta table from Azure.
Supported options for Azure are available `here
`__.
Following type of table paths are supported,
* az:///
* adl:///
* abfs[s]:///
>>> table_path = "az://container/path/to/delta-table/"
>>> storage_options = {
... "AZURE_STORAGE_ACCOUNT_NAME": "AZURE_STORAGE_ACCOUNT_NAME",
... "AZURE_STORAGE_ACCOUNT_KEY": "AZURE_STORAGE_ACCOUNT_KEY",
... }
>>> pl.scan_delta(
... table_path, storage_options=storage_options
... ).collect() # doctest: +SKIP
Creates a scan for a Delta table with additional delta specific options.
In the below example, `without_files` option is used which loads the table without
file tracking information.
>>> table_path = "/path/to/delta-table/"
>>> delta_table_options = {"without_files": True}
>>> pl.scan_delta(
... table_path, delta_table_options=delta_table_options
... ).collect() # doctest: +SKIP
r r )!_init_credential_provider_builder)+_get_credentials_from_provider_expiry_awarer$ Nr z?cannot use credential_provider when passing a DeltaTable object)
table_pathr r r z7To make use of pyarrow_options, set use_pyarrow to True)DeltaProtocolError) MAX_SUPPORTED_READER_VERSION NOT_SUPPORTED_READER_VERSIONSUPPORTED_READER_FEATURESz&The table's minimum reader version is z5 but polars delta scanner only supports version 1 or z with these reader features: z)The table has set these reader features: z= but these are not yet supported by the polars delta scanner.c [ U5 S:X a
U [ / 5 4$ / n/ nU R 5 H0 u pEXA; a UR XE45 M UR XE45 M2 [ U5 [ U5 4$ )Nr )lenr itemsappend)schemapartition_columns main_schema hive_schemanamedtypes r(
_split_schema!scan_delta.._split_schema~ sz !Q &6":% % !<<>KD(""D=1""D=1 * k"F;$777r* z lakefs://zs3://insertignoreF) r8 r; cast_optionsmissing_columns
extra_columnshive_partitioningr r r )r8 r r9 z list[str]returnztuple[Schema, Schema])$_check_if_delta_availabler r ,polars.io.cloud.credential_provider._builderr, .polars.io.cloud.credential_provider._providersr-
isinstance
ValueErrorbuild_credential_provider_get_delta_lake_table_storage_optionsto_pyarrow_datasetr deltalake.exceptionsr/ deltalake.tabler0 r1 r2 protocolmin_reader_versionreader_features
differencer5 r8 r metadatar9 file_uris table_uri
startswithreplacer r _default_iceberg) r# r r r r r r! r credential_provider_credsr r, r- credential_provider_buildermsgproviderdl_tblpa_dsr/ r0 r1 r2 table_protocolmissing_features delta_schema
polars_schemar9 r> r: r; rX file_uris r( r$ r$ s/ @ " $ f ) )&G ,'
#
(-@F-JOo &*#"/IIKKK
8 A GR " # *.I.U
E%2 D*C D / F &%% +/JXf 5 5 ;X AVTVX)/R))N=O PM NC$S) )==?L<(M );;88+48 8" -]=0.8.0. Read the documentation
`here `_.
NzWhen supplying a DeltaTable directly, `version`, `storage_options`, and `delta_table_options` are ignored.
To silence this warning, don't supply those parameters. )
stacklevel)r r r ) rH rK r r anywarningswarnRuntimeWarningrn rj r load_as_version)r. r r r rm ra s r( rN rN s *i 2 233 t# t+#4/
MMK
" *:6L gX / /%%
+
"
M %%
+
"
w' Mr* c 4 [ ( d
Sn [ U 5 eg )Nz=deltalake is not installed
Please run: pip install deltalake)r ModuleNotFoundError)r_ s r( rH rH s O!#&& r* c d [ U 6 n[ [ 1nX- =n( a SU< 3n[ U5 eg )Nz+dataframe contains unsupported data types: )r
r r TypeError)dtypes
schema_dtypesunsupported_typesoverlapr_ s r( _check_for_unsupported_typesr sA !6*M t
33w3;G;Gn 4r* )r# str | Path | DeltaTabler int | str | datetime | Noner zlist[str] | Noner bool | Noner dict[str, Any] | Noner 3CredentialProviderFunction | Literal['auto'] | Noner r r boolr! r rG r )r# r r r r r r r r r r r r! r r r rG r )rY z
str | Pathrh r rG rj )NNN)
r. r r r r r r r rG zdeltalake.DeltaTable)rG None)r{ zlist[DataType]rG r ))
__future__r rs r pathlibr typingr r polars.datatypesr r polars.datatypes.convertr
polars.dependenciesr r polars.io.cloud._utilsr
polars.io.parquetr #polars.io.pyarrow_dataset.functionsr #polars.io.scan_options.cast_optionsr
polars.schemar r r polarsr r r polars.io.cloudr r) r$ rn rN rH r rF r* r( r s " % ' 2 ? 3 * D ? $55: ,0 $ -1OU15 -1H #H )H H
H +
H MH /H H +H H\ ,0-1OU15 -1 ~ #~ )~ + ~
M ~ /
~ ~ +~ ~ ~B FJ ,0-115 1'1
(1 +1 / 1
1h' r*