Ë
|
Kñúh ã ó: d Z ddlmZ ddlmZ dgZ G d d« Zy)a$
|
A buffered iterator for big arrays.
|
|
This module solves the problem of iterating over a big file-based array
|
without having to read it into memory. The `Arrayterator` class wraps
|
an array object, and when iterated it will return sub-arrays with at most
|
a user-specified number of elements.
|
|
é )Úreduce)ÚmulÚ Arrayteratorc óV e Zd ZdZdZd dZd Zd Zd dZed « Z ed « Z
|
d
|
Z y) r aï
|
Buffered iterator for big arrays.
|
|
`Arrayterator` creates a buffered iterator for reading big arrays in small
|
contiguous blocks. The class is useful for objects stored in the
|
file system. It allows iteration over the object *without* reading
|
everything in memory; instead, small blocks are read and iterated over.
|
|
`Arrayterator` can be used with any object that supports multidimensional
|
slices. This includes NumPy arrays, but also variables from
|
Scientific.IO.NetCDF or pynetcdf for example.
|
|
Parameters
|
----------
|
var : array_like
|
The object to iterate over.
|
buf_size : int, optional
|
The buffer size. If `buf_size` is supplied, the maximum amount of
|
data that will be read into memory is `buf_size` elements.
|
Default is None, which will read as many element as possible
|
into memory.
|
|
Attributes
|
----------
|
var
|
buf_size
|
start
|
stop
|
step
|
shape
|
flat
|
|
See Also
|
--------
|
numpy.ndenumerate : Multidimensional array iterator.
|
numpy.flatiter : Flat array iterator.
|
numpy.memmap : Create a memory-map to an array stored
|
in a binary file on disk.
|
|
Notes
|
-----
|
The algorithm works by first finding a "running dimension", along which
|
the blocks will be extracted. Given an array of dimensions
|
``(d1, d2, ..., dn)``, e.g. if `buf_size` is smaller than ``d1``, the
|
first dimension will be used. If, on the other hand,
|
``d1 < buf_size < d1*d2`` the second dimension will be used, and so on.
|
Blocks are extracted along this dimension, and when the last block is
|
returned the process continues from the next dimension, until all
|
elements have been read.
|
|
Examples
|
--------
|
>>> import numpy as np
|
>>> a = np.arange(3 * 4 * 5 * 6).reshape(3, 4, 5, 6)
|
>>> a_itor = np.lib.Arrayterator(a, 2)
|
>>> a_itor.shape
|
(3, 4, 5, 6)
|
|
Now we can iterate over ``a_itor``, and it will return arrays of size
|
two. Since `buf_size` was smaller than any dimension, the first
|
dimension will be iterated over first:
|
|
>>> for subarr in a_itor:
|
... if not subarr.all():
|
... print(subarr, subarr.shape) # doctest: +SKIP
|
>>> # [[[[0 1]]]] (1, 1, 1, 2)
|
|
z numpy.libNc óä || _ || _ |j D cg c] }d c}| _ t |j « | _ |j D cg c] }d c}| _ y c c}w c c}w )Nr é )ÚvarÚbuf_sizeÚshapeÚstartÚlistÚstopÚstep)Úselfr r
|
Údims úOH:\Change_password\venv_build\Lib\site-packages\numpy/lib/_arrayterator_impl.pyÚ__init__zArrayterator.__init__X sS ØØ à#&§9¡9Ö-CaÒ-
|
ÜO Ø"%§)¡)Ö,3QÒ, ùò .ùâ,s A(Á A-c ó. t | j |« S ©N)Úgetattrr )r Úattrs r Ú __getattr__zArrayterator.__getattr__` s Ütxx Ó&Ð&ó c ó® t |t « s|f}g }t |« | j }}|D ]{ }|t u r0|j t d« g||z
|
dz z « t |« };t |t « r |j t ||dz d« « k|j |« } t |« }t |« |k r |t d« f|t |« z
|
z z }| j | j | j « }t t | j | j | j |« « D ] \ }\ }} }
|
}||j xs dz |j |< |
|
|j xs dz |j |< ||j xs | |z
|
z |j |< t# | |j | « |j |< |S )z-
|
Return a new arrayterator.
|
|
Nr r )Ú
|
isinstanceÚtupleÚlenÚndimÚEllipsisÚextendÚsliceÚintÚappendÚ __class__r r
|
Ú enumerateÚzipr r r Úmin) r ÚindexÚfixedÚlengthÚdimsÚslice_ÚoutÚir r r s r Ú __getitem__zArrayterator.__getitem__c s ô %¤Ô'ØHEØÜ5z 4§9¡9Øò %FØÑ!Ø e Dk]¨d°V©m¸aÑ.?Ñ @ÔAÜ UÜF¤CÔ(Ø U 6¨6°A©:°qÓ 9Õ:à VÕ$ð %ô e Ü u:Ò Ø eDk^ t¬c°%«jÑ'8Ñ9Ñ 9Eð nnTXX t§}¡}Ó5Ü.7ÜDJJ § ¡ ¨4¯9©9°eÓ<ó/>ò 1Ñ *AÑ*tT 6à F§L¡LÒ$5°AÑ6CIIaLØ &§+¡+Ò"2°Ñ3CHHQKØ 6§;¡;Ò#>°$¸±,Ñ?CHHQKÜ d C§H¡H¨Q¡KÓ0CHHQKð 1ð
|
r c ó t d t | j | j | j « D « « }| j
|
| S )z-
|
Return corresponding data.
|
|
c 3 ó, K | ] }t | y wr ©r! ©Ú.0Úts r ú <genexpr>z)Arrayterator.__array__.<locals>.<genexpr> s è ø ò 3 Quayñ 3ùó )r r& r r r r )r ÚdtypeÚcopyr, s r Ú __array__zArrayterator.__array__ sA ô
|
ñ 3¬#Ø
|
|
DII t§y¡yó+2ô 3ó 3àxxÑ Ð r c # óF K | D ] }|j E d{ y7 w)aX
|
A 1-D flat iterator for Arrayterator objects.
|
|
This iterator returns elements of the array to be iterated over in
|
`~lib.Arrayterator` one by one.
|
It is similar to `flatiter`.
|
|
See Also
|
--------
|
lib.Arrayterator
|
flatiter
|
|
Examples
|
--------
|
>>> a = np.arange(3 * 4 * 5 * 6).reshape(3, 4, 5, 6)
|
>>> a_itor = np.lib.Arrayterator(a, 2)
|
|
>>> for subarr in a_itor.flat:
|
... if not subarr:
|
... print(subarr, type(subarr))
|
...
|
0 <class 'numpy.int64'>
|
|
N)Úflat)r Úblocks r r< zArrayterator.flat s' è ø ð4 ò "EØ zz× !Ñ !ñ "Ø !ús ! !c óx t d t | j | j | j « D « « S )zk
|
The shape of the array to be iterated over.
|
|
For an example, see `Arrayterator`.
|
|
c 3 ó@ K | ] \ }}}||z
|
d z
|
|z d z yw)r N© )r4 r r r s r r6 z%Arrayterator.shape.<locals>.<genexpr>± s. è ø ò 7Ñ:K¸%ÀÀtte| aÑ'¨DÑ0°1Õ4ñ 7ùs )r r& r r r )r s r r zArrayterator.shape© s3 ô ñ 7ÜDJJ § ¡ ¨4¯9©9Ó5ô7ó 7ð 7r c # ó¦ K | j D cg c]
|
}|dk s | c}ry | j d d }| j d d }| j d d }| j j
|
} | j xs t t | j « }d}t |dz
|
dd« D ]| }|dk( r || dz ||< n8|| j | k r|| ||| z z ||< |}n| j | ||< t | j | || « ||< || j | z }~ t d t |||« D « « } | j | || ||< t |dz
|
dd« D ]J }|| | j | k\ s| j | ||< ||dz
|
xx | j |dz
|
z cc< L |d | j d k\ ry dc c}w w)Nr r éÿÿÿÿc 3 ó, K | ] }t | y wr r2 r3 s r r6 z(Arrayterator.__iter__.<locals>.<genexpr>Õ s è ø ÒE¨5 !9ÑEùr7 ) r r r r r r r
|
r r Úranger' r r& )
|
r r r r r ÚndimsÚcountÚrundimr. r, s
|
r Ú__iter__zArrayterator.__iter__´ sÛ è ø à ::Ö 2C¨°«CÓ 2Ø à
|
|
1 Øyy|Øyy|Ø àØMMÒ<¤V¬C°·±Ó%<Eð
|
FÜ5 19 b¨"Ó-ò /ð A:Ø # Ah¨lDGØdjj¨mÒ+à # Ah¨°°a±© Ñ 8DGØ Fð #ii¨lDGÜ dii¨l¨D°©GÓ4QØ §¡¨A¡ Ñ.ð /ô ÑE¬c°%¸¸tÓ.DÔEÓEFØ((6Ñ"Ò "ð ! LE&MÜ5 19 a¨Ó,ò 5Ø8tyy¨|Ó+Ø #zz¨!}E!HØ!a%L D§I¡I¨a°!©eÑ$4Ñ4Lð 5ð Qx499 Q<Ò'ØñE ùò 3ùs G
|
G G E GÆ AGr )NN) Ú__name__Ú
|
__module__Ú __qualname__Ú__doc__r r r/ r: Úpropertyr< r rH r@ r r r r sR ñCðJ Jó-ò'ò ó@ ð ñ"ó ð"ð8 ñ7ó ð7ó,r N)rL Ú functoolsr Úoperatorr Ú__all__r r@ r r ú<module>rQ s& ðñõ Ý à Ð
|
÷Pò Pr
|
