U )3g; @sddlmZmZddlmZmZz0ddlmZmZm Z m Z m Z m Z m Z mZmZWn8ek rZzedeeddW5dZ[XYnXzddlmZddlmZWn&ek rGdd d ZeZYnXdd d Zddd d efddZd efddZddZefddZdddZdS))Table RecordBatch) Expressionfield) DeclarationExecNodeOptionsTableSourceNodeOptionsFilterNodeOptionsProjectNodeOptionsAggregateNodeOptionsOrderByNodeOptionsHashJoinNodeOptionsAsofJoinNodeOptionsz@The pyarrow installation is not built with support for 'acero' ()N)ScanNodeOptionsc@s(eZdZGdddZGdddZdS)DatasetModuleStubc@s eZdZdS)zDatasetModuleStub.DatasetN__name__ __module__ __qualname__rr1/tmp/pip-unpacked-wheel-0fuzq0j6/pyarrow/acero.pyDataset3src@s eZdZdS)z!DatasetModuleStub.InMemoryDatasetNrrrrrInMemoryDataset6srN)rrrrrrrrrr2srTcCsltdt||d}dd|jjD}t|tdt|g}|jd}|dk rht|tdt|g}|S)Nscan use_threadscSsg|] }t|qSr)r).0frrr @sz$_dataset_to_decl..projectfilter) rrschemanames from_sequencer Z _scan_optionsgetr )datasetrdecl projectionsZ filter_exprrrr_dataset_to_decl;s r)Fc  st|ttjfs"tdt|t|ttjfsDtdt|it|ttfs\|g}t|D]\} } | | <qdit|ttfs|g}t|D]\} } | | <q|j j } |j j } |dks|dkrg} nX|dks|dkrg} nB|dks|dkr fdd | D} n|d kr&fd d | D} i}t| D]\} }||kr2| ||<q2i}t| D]\} }||kr\| ||<q\t|tjrt ||d }nt d t |}t|tjrt ||d }nt d t |}|rt|||| | |pd|pdd}nt||||pd|pdd}t d|||gd}|r`|dkr`t| }t| }t| }g}g}t| | D]\} }| t| kr||kr|||||}|tdt| t||gnx| |kr||krqdn^|r| |kr||kr||7}|r$| |kr$||kr$||7}|||t| qdt dt||}t ||g}|j|d }| tkrz|S| tjkrt|StddS)a\ Perform join of two tables or datasets. The result will be an output table with the result of the join operation Parameters ---------- join_type : str One of supported join types. left_operand : Table or Dataset The left operand for the join operation. left_keys : str or list[str] The left key (or keys) on which the join operation should be performed. right_operand : Table or Dataset The right operand for the join operation. right_keys : str or list[str] The right key (or keys) on which the join operation should be performed. left_suffix : str, default None Which suffix to add to left column names. This prevents confusion when the columns in left and right operands have colliding names. right_suffix : str, default None Which suffix to add to the right column names. This prevents confusion when the columns in left and right operands have colliding names. use_threads : bool, default True Whether to use multithreading or not. coalesce_keys : bool, default False If the duplicated keys should be omitted from one of the sides in the join result. output_type: Table or InMemoryDataset The output type for the exec plan result. Returns ------- result_table : Table or InMemoryDataset Expected Table or Dataset, got z left semiz left antiz right semiz right antiinnerz left outercsg|]}|kr|qSrrrcol)right_keys_orderrrrsz!_perform_join..z right outercsg|]}|kr|qSrrr,)left_keys_orderrrrsr table_source)Zoutput_suffix_for_leftZoutput_suffix_for_rightZhashjoinoptionsinputsz full outerZcoalescer Unsupported output typeN) isinstancerdsr TypeErrortypetuplelist enumerater"r#r)rrr setlenappendrZ_callZ_fieldr r$to_tabler)Z join_type left_operandZ left_keys right_operandZ right_keysZ left_suffixZ right_suffixrZ coalesce_keys output_typeidxkeyZ left_columns right_columnsZleft_column_keys_indicesZcolnameZright_column_keys_indices left_source right_source join_optsr'Zleft_columns_setZright_columns_setZright_operand_indexZprojected_col_namesr(r-Zright_key_indexZ projection result_tabler)r/r.r _perform_joinPs(                rKc sXt|ttjfs"tdt|t|ttjfsDtdt|t|ttfsX|g}tttfslgfdd|jj D} t |jj t | @} | rt d | t|tjrt ||d} ntdt|} t|tjrt ||d} ntdt|} t|||} td| | | gd}|j|d}|tkr6|S|tjkrLt|Std d S) a- Perform asof join of two tables or datasets. The result will be an output table with the result of the join operation Parameters ---------- left_operand : Table or Dataset The left operand for the join operation. left_on : str The left key (or keys) on which the join operation should be performed. left_by: str or list[str] The left key (or keys) on which the join operation should be performed. right_operand : Table or Dataset The right operand for the join operation. right_on : str or list[str] The right key (or keys) on which the join operation should be performed. right_by: str or list[str] The right key (or keys) on which the join operation should be performed. tolerance : int The tolerance to use for the asof join. The tolerance is interpreted in the same units as the "on" key. output_type: Table or InMemoryDataset The output type for the exec plan result. Returns ------- result_table : Table or InMemoryDataset r*csg|]}|gkr|qSrrr,right_byright_onrrr-sz&_perform_join_asof..zOColumns {} present in both tables. AsofJoin does not support column collisions.rr0Zasofjoinr2r5N)r6rr7rr8r9r:r;r"r#r= ValueErrorformatr)rrrr@r)rAZleft_onZleft_byrBrNrMZ tolerancerrCrFZcolumns_collisionsrGrHrIr'rJrrLr_perform_join_asofs`!       rQcCshd}t|trt|g}d}ttdt|dtdt|dg}|jdd}|rd| d}|S)a}Filter rows of a table based on the provided expression. The result will be an output table with only the rows matching the provided expression. Parameters ---------- table : Table or RecordBatch Table that should be filtered. expression : Expression The expression on which rows should be filtered. Returns ------- Table FTr0)r3r!rr) r6rrZ from_batchesrr$rr r@Zcombine_chunksZ to_batches)tableZ expressionZis_batchr'resultrrr _filter_tableWs   rTcKst|tjrt|dd}ntdt|}tdt|f|}t||g}|jdd}|t kr`|S|tj krtt |St ddS)NTrr0order_byr5) r6r7rr)rrr r$r@rrr8)Ztable_or_dataset sort_keysrCkwargsZ data_sourcerUr'rJrrr _sort_sourcews    rXc Cs2ttdt|tdt||dg}|j|dS)Nr0Z aggregate)keysr)rr$rr r@)rRZ aggregatesrYrr'rrr _group_bys  rZ)T)T)Z pyarrow.librrZpyarrow.computerrZpyarrow._acerorrrr r r r r r ImportErrorexcstrZpyarrow.datasetr&r7Zpyarrow._datasetrrr)rKrQrTrXrZrrrrs:0     4 V