U )3gZ@sPddlmZmZmZmZmZmZmZmZm Z m Z m Z m Z m Z mZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:m;Z;mZ>m?Z?m@Z@mAZAmBZBmCZCmDZDmEZEmFZFmGZGmHZHddlImJZJddlKZKddlLmMZMddlNZNddlOZPddlOmQZQddlRmSZSddZTeJd d ZUd d ZVd dZWddZXddZYddZZddZ[ddZ\ddZ]e]ddddfddZ^ddfddddZ_d dd!d"d#Z`d$d%Zadfddd&d'Zbdfddd(d)Zcd*ddd+d,d-Zdd.d/Zed0d1ZfdS)2)HFunctionFunctionOptionsFunctionRegistryHashAggregateFunctionHashAggregateKernelKernelScalarAggregateFunctionScalarAggregateKernelScalarFunction ScalarKernelVectorFunction VectorKernelArraySortOptionsAssumeTimezoneOptions CastOptions CountOptionsCumulativeOptionsCumulativeSumOptionsDayOfWeekOptionsDictionaryEncodeOptionsRunEndEncodeOptionsElementWiseAggregateOptionsExtractRegexOptions FilterOptions IndexOptions JoinOptionsListSliceOptionsListFlattenOptionsMakeStructOptionsMapLookupOptionsMatchSubstringOptions ModeOptions NullOptions PadOptionsPairwiseOptionsPartitionNthOptionsQuantileOptions RandomOptions RankOptionsReplaceSliceOptionsReplaceSubstringOptionsRoundBinaryOptions RoundOptionsRoundTemporalOptionsRoundToMultipleOptionsScalarAggregateOptionsSelectKOptionsSetLookupOptions SliceOptions SortOptions SplitOptionsSplitPatternOptionsStrftimeOptionsStrptimeOptionsStructFieldOptions TakeOptionsTDigestOptions TrimOptionsUtf8NormalizeOptionsVarianceOptions WeekOptions call_functionfunction_registry get_functionlist_functionscall_tabular_functionregister_scalar_functionregister_tabular_functionregister_aggregate_functionregister_vector_function UdfContext Expression) namedtupleN)dedent)_compute_docstrings) docscrapecCs|jjS)N)_doc arg_names)funcrQ3/tmp/pip-unpacked-wheel-0fuzq0j6/pyarrow/compute.py_get_arg_namesksrS_OptionsClassDoc)paramscCs"|js dSt|j}t|dS)NZ Parameters)__doc__rMZNumpyDocStringrT) options_classdocrQrQrR_scrape_options_class_docrs rYcCs|j}t|j|j|j|jd|_||_||_g}|j }|sZ|jdkrHdnd}d |j|}| |d|j }|r| |dt j|j} | tdt|} | D]8} |jd krd } nd } | | d | d | dq|dk rt|} | rH| jD]@}| |jd |jd |jD]}| d|d q(qnLtd|jdtt|}|jD] }| td |j|jqr| td|jd| td| dk r| d t| d d||_ |S)N)namearityrWoptions_required argumentsargumentz,Call compute function {!r} with the given {}z. z z. Parameters ---------- )Zvectorscalar_aggregatez Array-likezArray-like or scalar-likez :  z" Argument to compute function. z zOptions class z does not have a docstringz {0} : optional Parameter for {1} constructor. Either `options` or `{0}` can be passed, but not both at the same time. z& options : pyarrow.compute.zK, optional Alternative way of passing options. z memory_pool : pyarrow.MemoryPool, optional If not passed, will allocate memory from the default memory pool. z {} )!rNdictrZr[rWr\Z__arrow_compute_function____name__ __qualname__summaryformatappend descriptionrLZfunction_doc_additionsgetrKrSkindrYrUtypedescwarningswarnRuntimeWarninginspect signature parametersvaluesstripjoinrV)wrapperZ exposed_namerPrWZcpp_docZ doc_piecesrfZarg_strriZ doc_additionrOZarg_nameZarg_typeZoptions_class_docps options_sigrQrQrR_decorate_compute_functionysl           r{cCsJ|jj}|sdSz t|WStk rDtd|tYdSXdS)Nz!Python binding for {} not exposed)rNrWglobalsKeyErrorrnrorgrp)rP class_namerQrQrR_get_options_classs rcCsl|s|r(|dk rtd||||S|dk rht|trD|f|St||rR|Std||t|dS)NzMFunction {!r} called with both an 'options' argument and additional argumentsz-Function {!r} expected a {} parameter, got {}) TypeErrorrg isinstancercrl)rZrWoptionsargskwargsrQrQrR_handle_optionss*    rcs>dkr ddfdd }ndddfdd }|S)N memory_poolcsbtk r2t|kr2tddt|d|rTt|dtrTtt|S|d|S)N takes  positional argument(s), but were givenr)EllipsislenrrrI_calllistcall)rr)r[rP func_namerQrRrwsz&_make_generic_wrapper..wrapper)rrcstk rLt|kr2tddt|d|d}|d}nd}t|||}|rt|dtrtt||S|||S)NrrrrQr) rrrrrrIrrr)rrrrZ option_argsr[rPrrWrQrRrws  rQ)rrPrWr[rwrQrrR_make_generic_wrappers rcCsddlm}g}|D]}||||jq|D]}||||jq0|dk rt|}|jD]6}|j|j |j fks~t |r|j |j d}||qd||d|j dd||d|j ddt |S)Nr) Parameter)rkr)defaultr)rqrrhPOSITIONAL_ONLYVAR_POSITIONALrrrsrtrkPOSITIONAL_OR_KEYWORD KEYWORD_ONLYAssertionErrorreplace Signature)rO var_arg_namesrWrrUrZrzrxrQrQrR_make_signature s,      rcCsjt|}t|}|o |dd}|r8|dg}ng}t||||jd}t||||_t ||||S)N*)r[) rrS startswithpoplstriprr[r __signature__r{)rZrPrWrOZ has_varargrrwrQrQrR_wrap_function#s rcCst}t}ddd}|D]b}|||}||}|jdkrDq|jdkrZ|jdkrZq||ksjt|t||||<||<qdS)z Make global functions wrapping each compute function. Note that some of the automatically-generated wrappers may be overridden by custom versions below. and_or_)andorZhash_aggregater`rN) r|r@rBrjrArkr[rr)gregZrewritesZcpp_namerZrPrQrQrR_make_global_functions3s    rcCsh|dk p|dk }|r$|dk r$td|dkrXtjj|}|dkrNt|}n t|}td|g||S)a Cast array values to another data type. Can also be invoked as an array instance method. Parameters ---------- arr : Array-like target_type : DataType or str Type to cast to safe : bool, default True Check for overflows or other unsafe conversions options : CastOptions, default None Additional checks pass by CastOptions memory_pool : MemoryPool, optional memory pool to use for allocations during function execution. Examples -------- >>> from datetime import datetime >>> import pyarrow as pa >>> arr = pa.array([datetime(2010, 1, 1), datetime(2015, 1, 1)]) >>> arr.type TimestampType(timestamp[us]) You can use ``pyarrow.DataType`` objects to specify the target type: >>> cast(arr, pa.timestamp('ms')) [ 2010-01-01 00:00:00.000, 2015-01-01 00:00:00.000 ] >>> cast(arr, pa.timestamp('ms')).type TimestampType(timestamp[ms]) Alternatively, it is also supported to use the string aliases for these types: >>> arr.cast('timestamp[ms]') [ 2010-01-01 00:00:00.000, 2015-01-01 00:00:00.000 ] >>> arr.cast('timestamp[ms]').type TimestampType(timestamp[ms]) Returns ------- casted : Array The cast result as a new Array NzRMust either pass values for 'target_type' and 'safe' or pass a value for 'options'Fcast) ValueErrorpatypeslib ensure_typerZunsafesafer?)ZarrZ target_typerrrZsafe_vars_passedrQrQrRrSs6   rrcCs|dk r.|dk r"||||}qB||}n|dk rB|d|}t|tjs`tj||jd}n |j|jkrtj||jd}t|d}td|g||}|dk r|dkrtj||t d}|S)a Find the index of the first occurrence of a given value. Parameters ---------- data : Array-like value : Scalar-like object The value to search for. start : int, optional end : int, optional memory_pool : MemoryPool, optional If not passed, will allocate memory from the default memory pool. Returns ------- index : int the index, or -1 if not found Nrrlvalueindex) slicerrScalarscalarrlas_pyrr?Zint64)datarstartendrrresultrQrQrRrs     rT) boundscheckrcCst|d}td||g||S)a Select values (or records) from array- or table-like data given integer selection indices. The result will be of the same type(s) as the input, with elements taken from the input array (or record batch / table fields) at the given indices. If an index is null then the corresponding value in the output will be null. Parameters ---------- data : Array, ChunkedArray, RecordBatch, or Table indices : Array, ChunkedArray Must be of integer type boundscheck : boolean, default True Whether to boundscheck the indices. If False and there is an out of bounds index, will likely cause the process to crash. memory_pool : MemoryPool, optional If not passed, will allocate memory from the default memory pool. Returns ------- result : depends on inputs Selected values for the given indices Examples -------- >>> import pyarrow as pa >>> arr = pa.array(["a", "b", "c", None, "e", "f"]) >>> indices = pa.array([0, None, 4, 3]) >>> arr.take(indices) [ "a", null, "e", null ] )rtake)r9r?)rindicesrrrrQrQrRrs( rcCsVt|tjtjtjfs(tj||jd}n |j|jkrHtj||jd}td||gS)aeReplace each null element in values with a corresponding element from fill_value. If fill_value is scalar-like, then every null element in values will be replaced with fill_value. If fill_value is array-like, then the i-th element in values will be replaced with the i-th element in fill_value. The fill_value's type must be the same as that of values, or it must be able to be implicitly casted to the array's type. This is an alias for :func:`coalesce`. Parameters ---------- values : Array, ChunkedArray, or Scalar-like object Each null element is replaced with the corresponding value from fill_value. fill_value : Array, ChunkedArray, or Scalar-like object If not same type as values, will attempt to cast. Returns ------- result : depends on inputs Values with all null elements replaced Examples -------- >>> import pyarrow as pa >>> arr = pa.array([1, 2, None, 3], type=pa.int8()) >>> fill_value = pa.scalar(5, type=pa.int8()) >>> arr.fill_null(fill_value) [ 1, 2, 5, 3 ] >>> arr = pa.array([1, 2, None, 4, None]) >>> arr.fill_null(pa.array([10, 20, 30, 40, 50])) [ 1, 2, 30, 4, 50 ] rZcoalesce) rrArray ChunkedArrayrrrlrr?)rtZ fill_valuerQrQrR fill_nulls 3 rcCsR|dkr g}t|tjtjfr*|dntdd|}t||}td|g||S)a Select the indices of the top-k ordered elements from array- or table-like data. This is a specialization for :func:`select_k_unstable`. Output is not guaranteed to be stable. Parameters ---------- values : Array, ChunkedArray, RecordBatch, or Table Data to sort and get top indices from. k : int The number of `k` elements to keep. sort_keys : List-like Column key names to order by when input is table-like data. memory_pool : MemoryPool, optional If not passed, will allocate memory from the default memory pool. Returns ------- result : Array Indices of the top-k ordered elements Examples -------- >>> import pyarrow as pa >>> import pyarrow.compute as pc >>> arr = pa.array(["a", "b", "c", None, "e", "f"]) >>> pc.top_k_unstable(arr, k=3) [ 5, 4, 2 ] N)dummy descendingcSs|dfS)NrrQZkey_namerQrQrROz top_k_unstable..select_k_unstablerrrrrhmapr0r?rtk sort_keysrrrQrQrRtop_k_unstable%s%  rcCsR|dkr g}t|tjtjfr*|dntdd|}t||}td|g||S)a Select the indices of the bottom-k ordered elements from array- or table-like data. This is a specialization for :func:`select_k_unstable`. Output is not guaranteed to be stable. Parameters ---------- values : Array, ChunkedArray, RecordBatch, or Table Data to sort and get bottom indices from. k : int The number of `k` elements to keep. sort_keys : List-like Column key names to order by when input is table-like data. memory_pool : MemoryPool, optional If not passed, will allocate memory from the default memory pool. Returns ------- result : Array of indices Indices of the bottom-k ordered elements Examples -------- >>> import pyarrow as pa >>> import pyarrow.compute as pc >>> arr = pa.array(["a", "b", "c", None, "e", "f"]) >>> pc.bottom_k_unstable(arr, k=3) [ 0, 1, 2 ] N)r ascendingcSs|dfS)NrrQrrQrQrRr~rz#bottom_k_unstable..rrrrQrQrRbottom_k_unstableTs%  rsystem) initializerrrcCst|d}tdg|||dS)aB Generate numbers in the range [0, 1). Generated values are uniformly-distributed, double-precision in range [0, 1). Algorithm and seed can be changed via RandomOptions. Parameters ---------- n : int Number of values to generate, must be greater than or equal to 0 initializer : int or str How to initialize the underlying random generator. If an integer is given, it is used as a seed. If "system" is given, the random generator is initialized with a system-specific source of (hopefully true) randomness. Other values are invalid. options : pyarrow.compute.RandomOptions, optional Alternative way of passing options. memory_pool : pyarrow.MemoryPool, optional If not passed, will allocate memory from the default memory pool. )rrandom)length)r'r?)nrrrrQrQrRrs rcGsrt|}|dkrdt|dttfr0t|dSt|dtrLt|dStdt |dn t|SdS)aReference a column of the dataset. Stores only the field's name. Type and other information is known only when the expression is bound to a dataset having an explicit scheme. Nested references are allowed by passing multiple names or a tuple of names. For example ``('foo', 'bar')`` references the field named "bar" inside the field named "foo". Parameters ---------- *name_or_index : string, multiple strings, tuple or int The name or index of the (possibly nested) field the expression references to. Returns ------- field_expr : Expression Reference to the given field Examples -------- >>> import pyarrow.compute as pc >>> pc.field("a") >>> pc.field(1) >>> pc.field(("a", "b")) >> pc.field("a", "b") r?r@rArBrCrDrErFrGrHrI collectionsrJrqtextwraprKrnZpyarrowrrLZpyarrow.vendoredrMrSrTrYr{rrrrrrrrrrrrrrrrQrQrQrRs8)O     S E&,;//1