Ë
|
nñúhRl ã óp d Z ddlmZ ddlmZ ddlmZmZ ddlZddlm Z ddl
|
Z ddl m Z ddlmZmZmZmZmZ dd lmZmZ dd
|
lmZ dd lmZ ddlZ dd lm Z m Z m Z dd l m!Z! ddl"m#c m$c m%Z& ddl'm(Z( e r
|
ddl)m*Z*m+Z+m,Z, e d« Z- e d« Z.ddZ/ddZ0 G d d« Z1 G d de(ejd « Z3y)a
|
Read SAS7BDAT files
|
|
Based on code written by Jared Hobbs:
|
https://bitbucket.org/jaredhobbs/sas7bdat
|
|
See also:
|
https://github.com/BioStatMatt/sas7bdat
|
|
Partial documentation of the file format:
|
https://cran.r-project.org/package=sas7bdat/vignettes/sas7bdat.pdf
|
|
Reference for binary data compression:
|
http://collaboration.cmc.ec.gc.ca/science/rpn/biblio/ddj/Website/articles/CUJ/1992/9210/ross/ross.htm
|
é )Ú annotations)Úabc)ÚdatetimeÚ timedeltaN)Ú TYPE_CHECKING)Ú
|
get_option)Úread_double_with_byteswapÚread_float_with_byteswapÚread_uint16_with_byteswapÚread_uint32_with_byteswapÚread_uint64_with_byteswap)ÚParserÚget_subheader_index)Úcast_from_unit_vectorized)ÚEmptyDataError)Ú DataFrameÚ TimestampÚisna)Ú
|
get_handle)Ú
|
ReaderBase)ÚCompressionOptionsÚFilePathÚ
|
ReadBufferz
|
1970-01-01z
|
1960-01-01c óÊ t | « rt j S |dk( rt ddd« t | ¬« z S |dk( rt ddd« t | ¬« z S t d« )NÚsé¨ é )ÚsecondsÚd)Údayszunit must be 'd' or 's')r ÚpdÚNaTr r Ú
|
ValueError)Ú sas_datetimeÚunits úIH:\Change_password\venv_build\Lib\site-packages\pandas/io/sas/sas7bdat.pyÚ_parse_datetimer' C sa Ü LÔÜvv à s{Üa Ó#¤i¸ Ô&EÑEÐEà Üa Ó#¤i°\Ô&BÑBÐBô Ð2Ó3Ð3ó c óZ t t z
|
j d« }|dk( rNt | j dd¬« }|j d« |z }t j || j d¬« S t j | d¬« |z }t j |d | j d¬
|
« S ) aÎ
|
Convert to Timestamp if possible, otherwise to datetime.datetime.
|
SAS float64 lacks precision for more than ms resolution so the fit
|
to datetime.datetime is ok.
|
|
Parameters
|
----------
|
sas_datetimes : {Series, Sequence[float]}
|
Dates or datetimes in SAS
|
unit : {'d', 's'}
|
"d" if the floats represent dates, "s" for datetimes
|
|
Returns
|
-------
|
Series
|
Series of datetime64 dtype or datetime.datetime.
|
r Úms)r% Úout_unitzM8[ms]F©ÚindexÚcopyzM8[D]©ÚdtypezM8[s]©r0 r- r. ) Ú _sas_originÚ _unix_originÚas_unitr Ú_valuesÚviewr! ÚSeriesr- ÚnpÚarray)Ú sas_datetimesr% ÚtdÚmillisÚdt64msÚvalss r& Ú_convert_datetimesr? Q s ô$ Ñ
|
$× -Ñ -¨cÓ 2BØ s{Ü*Ø × !Ñ !¨°dô
|
ð XÓ&¨Ñ+Üyy }×':Ñ':ÀÔGÐGäxx ¨WÔ5¸Ñ:Üyy W°M×4GÑ4GÈeÔTÐTr( c ón e Zd ZU ded< ded< ded< ded< ded< ded < d d
|
Zy ) Ú_ColumnÚintÚcol_idú str | bytesÚnameÚlabelÚformatÚbytesÚctypeÚlengthc óX || _ || _ || _ || _ || _ || _ y ©N)rC rE rF rG rI rJ )ÚselfrC rE rF rG rI rJ s r& Ú__init__z_Column.__init__w s. ð Ø Ø
|
Ø Ø
|
Ø r( N)rC rB rE rD rF rD rG rD rI rH rJ rB ÚreturnÚNone)Ú__name__Ú
|
__module__Ú __qualname__Ú__annotations__rN © r( r& rA rA o sl
Ø KØ
|
ÓØ ÓØ ÓØ LØ Kð àð ð ð ð
|
ð ð ð ð ð ð ð ð
|
ô r( rA c ó8 e Zd ZU dZded< ded< d# d$dZd%dZd%d Zd%d
|
Zd&d Z d&d Z
|
d'd Z d(dZ d)dZ d*dZd+dZd&dZd,dZd&dZd&dZd-dZd-dZd-dZd-dZd-dZd-dZd-d Zd-d Zd.d/d Z d Z d'd Z d! Z d0d"Z y)1ÚSAS7BDATReaderaË
|
Read SAS files in SAS7BDAT format.
|
|
Parameters
|
----------
|
path_or_buf : path name or buffer
|
Name of SAS file or file-like object pointing to SAS file
|
contents.
|
index : column identifier, defaults to None
|
Column to use as index.
|
convert_dates : bool, defaults to True
|
Attempt to convert dates to Pandas datetime values. Note that
|
some rarely used SAS date formats may be unsupported.
|
blank_missing : bool, defaults to True
|
Convert empty strings to missing values (SAS uses blanks to
|
indicate missing character variables).
|
chunksize : int, defaults to None
|
Return SAS7BDATReader object for iterations, returns chunks
|
with given number of lines.
|
encoding : str, 'infer', defaults to None
|
String encoding acc. to Python standard encodings,
|
encoding='infer' tries to detect the encoding from the file header,
|
encoding=None will leave the data in binary format.
|
convert_text : bool, defaults to True
|
If False, text variables are left as raw bytes.
|
convert_header_text : bool, defaults to True
|
If False, header text, including column names, are left as raw
|
bytes.
|
rB Ú _int_lengthz bytes | NoneÚ _cached_pageNc
|
óÆ || _ || _ || _ || _ || _ || _ || _ d| _ d| _ g | _ g | _
|
g | _ g | _ g | _ d | _ g | _ g | _ g | _ d| _ d| _ d| _ t) |dd| ¬« | _ | j* j, | _ | j0 | j2 | j4 | j6 | j8 | j: | j< | j> d g | _ | jC « | jE « y # tF $ r | jI « w xY w)Nzlatin-1r( r ÚrbF)Úis_textÚ compression)%r- Ú convert_datesÚ blank_missingÚ chunksizeÚencodingÚ convert_textÚconvert_header_textÚdefault_encodingr] Úcolumn_names_rawÚ column_namesÚcolumn_formatsÚcolumnsÚ%_current_page_data_subheader_pointersrY Ú_column_data_lengthsÚ_column_data_offsetsÚ _column_typesÚ_current_row_in_file_indexÚ_current_row_on_page_indexr ÚhandlesÚhandleÚ _path_or_bufÚ_process_rowsize_subheaderÚ _process_columnsize_subheaderÚ_process_subheader_countsÚ _process_columntext_subheaderÚ _process_columnname_subheaderÚ#_process_columnattributes_subheaderÚ_process_format_subheaderÚ _process_columnlist_subheaderÚ_subheader_processorsÚ_get_propertiesÚ_parse_metadataÚ ExceptionÚclose)
|
rM Ú path_or_bufr- r^ r_ r` ra rb rc r] s
|
r& rN zSAS7BDATReader.__init__¬ s` ð
|
Ø *ÔØ *ÔØ"Ø Ø (ÔØ#6Ô à )Ô Ø ÔØ-/Ô Ø/1ÔØ13ÔØ&( àLNÔ2Ø ÔØ/1Ô!Ø/1Ô!Ø*,Ôà*+Ô'Ø*+Ô'Ø*+Ô'ä!Ø u¸+ô
|
ð !LL× /Ñ /Ôð × +Ñ +Ø × .Ñ .Ø × *Ñ *Ø × .Ñ .Ø × .Ñ .Ø × 4Ñ 4Ø × *Ñ *Ø × .Ñ .Ø ð
|
&
|
Ô"ð Ø × Ñ Ô "Ø × Ñ Õ "øÜò Ø JJLØ ð ús Ä$ E ÅE c ó` t j | j t j ¬« S )z5Return a numpy int64 array of the column data lengthsr/ )r8 Úasarrayrj Úint64©rM s r& Úcolumn_data_lengthsz"SAS7BDATReader.column_data_lengthsë ó äzz$×3Ñ3¼2¿8¹8ÔDÐDr( c ó` t j | j t j ¬« S )z0Return a numpy int64 array of the column offsetsr/ )r8 r rk r r s r& Úcolumn_data_offsetsz"SAS7BDATReader.column_data_offsetsï r
r( c ój t j | j t j d« ¬« S )zj
|
Returns a numpy character array of the column types:
|
s (string) or d (double)
|
ÚS1r/ )r8 r rl r0 r s r& Ú column_typeszSAS7BDATReader.column_typesó s# ô
|
zz$×,Ñ,´B·H±H¸T³NÔCÐCr( c ó8 | j j « y rL )ro r~ r s r& r~ zSAS7BDATReader.closeú s Ø ×ÑÕ r( c ó | j j d« | j j d« | _ | j dt t
|
j « t
|
j k7 r t d« | j t
|
j t
|
j « }|t
|
j k( r9d| _ d| _ t
|
j | _ t
|
j | _ n8d| _ t
|
j$ | _ t
|
j& | _ d| _ | j t
|
j( t
|
j* « }|t
|
j, k( rt
|
j. }nd}| j t
|
j0 t
|
j2 « }|dk( r d | _ t6 j8 d
|
k( | _ n d | _ t6 j8 d k( | _ | j t
|
j< t
|
j> « d }|t
|
j@ v r9t
|
j@ | | _! | jD d k( r | jB | _" n d| d| _! tG ddd« }| jI t
|
jJ |z t
|
jL « }|tO jP |d¬« z | _) | jI t
|
jT |z t
|
jV « }|tO jP |d¬« z | _, | j[ t
|
j\ |z t
|
j^ « | _0 | j j | j` dz
|
« }| xj |z c_ t | j « | j` k7 r t d« | j[ t
|
jb |z t
|
jd « | _3 y )Nr i z'magic number mismatch (not a SAS file?)Té Fé ó ú<Úbigú>ÚlittleÚinferzunknown (code=ú)r r r )r% z*The SAS7BDAT file appears to be truncated.)4rq ÚseekÚreadrY ÚlenÚconstÚmagicr# Ú _read_bytesÚalign_1_offsetÚalign_1_lengthÚu64_byte_checker_valueÚU64rX Úpage_bit_offset_x64Ú_page_bit_offsetÚ subheader_pointer_length_x64Ú_subheader_pointer_lengthÚpage_bit_offset_x86Ú subheader_pointer_length_x86Úalign_2_offsetÚalign_2_lengthÚalign_1_checker_valueÚ align_2_valueÚendianness_offsetÚendianness_lengthÚ
|
byte_orderÚsysÚ byteorderÚ need_byteswapÚencoding_offsetÚencoding_lengthÚencoding_namesÚinferred_encodingra r Ú _read_floatÚdate_created_offsetÚdate_created_lengthr! Ú to_timedeltaÚ date_createdÚdate_modified_offsetÚdate_modified_lengthÚ date_modifiedÚ
|
_read_uintÚheader_size_offsetÚheader_size_lengthÚ header_lengthÚpage_size_offsetÚpage_size_lengthÚ _page_length)rM ÚbufÚalign1ÚepochÚxs r& r{ z SAS7BDATReader._get_propertiesý s à ×Ñ× Ñ qÔ!Ø × -Ñ -× 2Ñ 2°3Ó 7ÔØ × Ñ Q¤¤U§[¡[Ó!1Ð 2´e·k±kÒ AÜ Ð FÓGÐ Gð × Ñ u× 3Ñ 3´U×5IÑ5IÓJØ %×.Ñ.Ò .ØDHØ DÔ Ü$)×$=Ñ$=DÔ !Ü-2×-OÑ-ODÕ *à DHÜ$)×$=Ñ$=DÔ !Ü-2×-OÑ-ODÔ *Ø DÔ Ø× Ñ u× 3Ñ 3´U×5IÑ5IÓJØ %×-Ñ-Ò -Ü×(Ñ(FàFð × Ñ u× 6Ñ 6¼×8OÑ8OÓPØ '>Ø !DOÜ!$§ ¡ °%Ñ!7DÕ à !DOÜ!$§ ¡ °(Ñ!:DÔ ð × Ñ u× 4Ñ 4´e×6KÑ6KÓLÈQÑOØ %×&Ñ&Ñ &Ü%*×%9Ñ%9¸#Ñ%>DÔ "Ø}} Ò'Ø $× 6Ñ 6 à'5°c°U¸!Ð%<DÔ "ô q !Ó$Ø × Ñ Ü × %Ñ %¨Ñ .´×0IÑ0Ió
|
ð "¤B§O¡O°A¸CÔ$@Ñ @ÔØ × Ñ Ü × &Ñ &¨Ñ /´×1KÑ1Kó
|
ð #¤R§_¡_°Q¸SÔ%AÑ AÔà !__Ü × $Ñ $ vÑ -¬u×/GÑ/Gó
|
Ôð
|
× Ñ ×$Ñ$ T×%7Ñ%7¸#Ñ%=Ó>Ø ×ÒSÑ Õô t× Ñ Ó ! T×%7Ñ%7Ò 7Ü Ð IÓJÐ Jà OOÜ × "Ñ " VÑ +¬U×-CÑ-Có
|
Õr( c ó | j | j xs d¬« }|j r| j « t |S )Nr )Únrows)r r` Úemptyr~ Ú StopIteration)rM Údas r& Ú__next__zSAS7BDATReader.__next__C s6 Ø YYT^^Ò 0¨qYÓ 1Ø 88Ø JJLÜ Ð Ø r( c óì | j J |dk( r!t | j || j « S |dk( r!t | j || j « S | j « t d« )Nr r zinvalid float width)rY r
|
r¯ r r~ r# ©rM ÚoffsetÚwidths r& r´ zSAS7BDATReader._read_floatK s{ Ø× Ñ Ð,Ð,Ð,Ø A:Ü+Ø×!Ñ! 6¨4×+=Ñ+=óð ð aZÜ,Ø×!Ñ! 6¨4×+=Ñ+=óð ð JJLÜ Ð 2Ó3Ð 3r( c ól | j J |dk( r| j |d« d S |dk( r!t | j || j « S |dk( r!t | j || j « S |dk( r!t | j || j « S | j « t d« )Nr r é r r zinvalid int width)rY r r r¯ r r r~ r# rÎ s r& r¼ zSAS7BDATReader._read_uintZ sà Ø× Ñ Ð,Ð,Ð,Ø A:Ø×#Ñ# F¨AÓ.¨qÑ1Ð 1Ø aZÜ,Ø×!Ñ! 6¨4×+=Ñ+=óð ð aZÜ,Ø×!Ñ! 6¨4×+=Ñ+=óð ð aZÜ,Ø×!Ñ! 6¨4×+=Ñ+=óð ð JJLÜ Ð 0Ó1Ð 1r( c ó® | j J ||z t | j « kD r| j « t d« | j |||z S )Nz The cached page is too small.)rY r r~ r# ©rM rÏ rJ s r& r zSAS7BDATReader._read_bytesn sW Ø× Ñ Ð,Ð,Ð,Ø F?S ×!2Ñ!2Ó 3Ò 3Ø JJLÜ Ð <Ó=Ð =Ø× Ñ ¨&°6©/Ð:Ð:r( c ób | j | j ||« j d« « S )Nó )Ú_convert_header_textr ÚrstriprÔ s r& Ú _read_and_convert_header_textz,SAS7BDATReader._read_and_convert_header_textu s0 Ø×(Ñ(Ø × Ñ V VÓ ,× 3Ñ 3°HÓ =ó
|
ð
|
r( c ó d}|s| j j | j « | _ t | j « dk ry t | j « | j k7 r t d« | j « }|sy y )NFr z2Failed to read a meta data page from the SAS file.)rq r r rY r r# Ú_process_page_meta)rM Údones r& r| z SAS7BDATReader._parse_metadataz sw ØÙØ $× 1Ñ 1× 6Ñ 6°t×7HÑ7HÓ IDÔ Ü4×$Ñ$Ó%¨Ò*ØÜ4×$Ñ$Ó%¨×):Ñ):Ò:Ü Ð!UÓVÐVØ×*Ñ*Ó,Dô r( c ót | j « t j t j t j gz }| j
|
|v r| j « | j
|
t j k( }| j
|
t j k( }t |xs |xs | j g k7 « S rL )
|
Ú_read_page_headerr Úpage_meta_typesÚ page_amd_typeÚ page_mix_typeÚ_current_page_typeÚ_process_page_metadataÚpage_data_typeÚboolri )rM ÚptÚ is_data_pageÚ is_mix_pages r& rÛ z!SAS7BDATReader._process_page_meta s¢ Ø × Ñ Ô Ü × "Ñ "¤e×&9Ñ&9¼5×;NÑ;NÐ%OÑ OØ × "Ñ " bÑ (Ø × 'Ñ 'Ô )Ø×.Ñ.´%×2FÑ2FÑF Ø×-Ñ-´×1DÑ1DÑD ÜØ ò @Øò @à×9Ñ9¸RÑ?ó
|
ð
|
r( c ó | j }t j |z }| j |t j « t j
|
z | _ t j |z }| j |t j « | _ t j |z }| j |t j « | _ y rL ) r¡ r Úpage_type_offsetr¼ Úpage_type_lengthÚpage_type_mask2râ Úblock_count_offsetÚblock_count_lengthÚ_current_page_block_countÚsubheader_count_offsetÚsubheader_count_lengthÚ _current_page_subheaders_count)rM Ú
|
bit_offsetÚtxs r& rÞ z SAS7BDATReader._read_page_header s Ø×*Ñ*
|
Ü × #Ñ # jÑ 0à OOB¤× 6Ñ 6Ó 7¼%×:OÑ:OÑ Oð Ô ô × %Ñ %¨
|
Ñ 2Ø)-¯ © ¸¼U×=UÑ=UÓ)VÔ&Ü × )Ñ )¨JÑ 6Ø.2¯o©oØ ×,Ñ,ó/
|
Õ+r( c ó8 | j }t | j « D ]u }t j |z }|| j
|
|z z }| j || j « }|| j z }| j || j « }|| j z }| j |d« }|dz }| j |d« }|dk( s|t j k( rÁ| j || j « } t | « }
|
| j |
|
} | t|t j dfv } |t j k( } | j r#| r!| r | j j! ||f« O| j# « t% d| « | ||« x y )Nr r z Unknown subheader signature )r¡ Úrangerò r Úsubheader_pointers_offsetr£ r¼ rX Útruncated_subheader_idr r rz Úcompressed_subheader_idÚcompressed_subheader_typer] ri Úappendr~ r# )rM ró ÚirÏ Ú total_offsetÚsubheader_offsetÚsubheader_lengthÚsubheader_compressionÚsubheader_typeÚsubheader_signatureÚsubheader_indexÚsubheader_processorÚf1Úf2s r& rã z%SAS7BDATReader._process_page_metadata s Ø×*Ñ*
|
ät×:Ñ:Ó;ó & HAÜ×4Ñ4°zÑAFØ! D×$BÑ$BÀQÑ$FÑFLà #¨|¸T×=MÑ=MÓ NÐ Ø D× ,Ñ ,Ñ ,Là #¨|¸T×=MÑ=MÓ NÐ Ø D× ,Ñ ,Ñ ,Là$(§O¡O°LÀ!Ó$DÐ !Ø AÑ Là !__¨\¸1Ó =Nð ! AÒ%Ø(¬E×,HÑ,HÒHàà"&×"2Ñ"2Ð3CÀT×EUÑEUÓ"VÐ Ü 1Ð2EÓ FOØ"&×"<Ñ"<¸_Ñ"MÐ à"Ð*Ø*¬u×/LÑ/LÈaÐ.PÐPØ#¤u×'FÑ'FÑFØ×#Ò#©©rØ×>Ñ>×EÑEØ)Ð+;Ð<öð JJLÜ$Ø6Ð7JÐ6KÐLóð ñ $Ð$4Ð6FÖGñM& Hr( c ót | j }|}|}| j r |dz }|dz }n
|
|dz }|dz }| j |t j |z z |« | _ | j |t j |z z |« | _ | j |t j |z z |« | _ | j |t j |z z |« | _ t j |z }| j ||z |« | _ | j |d« | _ | j |d« | _ y )Niª i ib iz rÒ )rX r r¼ r Ú row_length_offset_multiplierÚ
|
row_lengthÚrow_count_offset_multiplierÚ row_countÚcol_count_p1_multiplierÚ col_count_p1Úcol_count_p2_multiplierÚ col_count_p2Ú'row_count_on_mix_page_offset_multiplierÚ_mix_page_row_countÚ_lcsÚ_lcp)rM rÏ rJ Úint_lenÚ
|
lcs_offsetÚ
|
lcp_offsetÚmxs r& rr z)SAS7BDATReader._process_rowsize_subheaderÉ s1 Ø×"Ñ"Ø
|
Ø
|
Ø 88Ø #Ñ JØ #Ñ Jà #Ñ JØ #Ñ Jà //Ø U×7Ñ7¸'ÑAÑ AØ ó
|
ð Ø U×6Ñ6¸Ñ@Ñ @Ø ó
|
ð !OOØ U×2Ñ2°WÑ<Ñ <¸gó
|
Ôð !OOØ U×2Ñ2°WÑ<Ñ <¸gó
|
Ôô × :Ñ :¸WÑ DØ#'§?¡?°6¸B±;ÀÓ#HÔ ØOO J°Ó2 ØOO J°Ó2 r( c ó | j }||z }| j ||« | _ | j | j z | j k7 r4t d| j d| j d| j d« y y )Nz Warning: column count mismatch (z + z != z)
|
)rX r¼ Ú column_countr r Úprint)rM rÏ rJ r s r& rs z,SAS7BDATReader._process_columnsize_subheaderç s Ø×"Ñ"Ø'ÑØ OO¨F°GÓ <ÔØ × Ñ t× 0Ñ 0Ñ 0°D×4EÑ4EÒ EÜ Ø2°4×3DÑ3DÐ2EÀSØ×$Ñ$Ð% T¨$×*;Ñ*;Ð)<¸CðAõ ð Fr( c ó y rL rU rÔ s r& rt z(SAS7BDATReader._process_subheader_countsò s Ø r( c ór || j z }| j |t j « }| j ||« }|d| j d« }| j j |« t | j « dk( r®d}t j D ] }||v s|} || _
|
|| j z }|dz }| j r|dz }| j || j « }|j d« }|dk( rNd| _ |dz }| j r|dz }| j || j « }|d| j | _ n¶|t j k( rG|d z }| j r|dz }| j || j « }|d| j | _ n\| j dkD rMd| _ |dz }| j r|dz }| j || j « }|d| j | _ t! | d
|
« r!| j# | j « | _ y y y ) Nr rÖ r r( é r ó é é( Ú creator_proc)rX r¼ r Útext_block_size_lengthr rØ re rû r Úcompression_literalsr] r r r r! Úrle_compressionÚhasattrr× ) rM rÏ rJ Útext_block_sizerÃ Ú cname_rawÚcompression_literalÚclÚoffset1s r& ru z,SAS7BDATReader._process_columntext_subheaderõ s Ø$×"Ñ"Ñ"Ø //¨&´%×2NÑ2NÓOà× Ñ v Ó7Ø/Ð*×1Ñ1°(Ó; Ø × Ñ ×$Ñ$ YÔ/ä t×$Ñ$Ó %¨Ó *Ø"%Ð Ü×0Ñ0ò -Ø?Ø*,Ñ'ð -ð 3DÔ Ø d×&Ñ&Ñ &Fà rkGØxxØ1 à×"Ñ" 7¨D¯I©IÓ6CØ"%§*¡*¨WÓ"5Ð Ø" cÒ)Ø Ø 2+Ø88ØqLGØ×&Ñ& w°· ± Ó:Ø$'¨¨D¯I©IÐ$6Õ!Ø$¬×(=Ñ(=Ò=Ø 2+Ø88ØqLGØ×&Ñ& w°· ± Ó:Ø$'¨¨D¯I©IÐ$6Õ!ØQ Ø Ø 2+Ø88ØqLGØ×&Ñ& w°· ± Ó:Ø$'¨¨D¯I©IÐ$6Ô!Üt^Ô,Ø$(×$=Ñ$=¸d×>OÑ>OÓ$PÕ!ð -ðE +r( c ó¤ | j }||z }|d|z z
|
dz
|
dz }t |« D ]" }|t j |dz z z t j z }|t j |dz z z t j
|
z }|t j |dz z z t j z }| j |t j « } | j |t j « }
|
| j |t j « } | j | } | |
|
|
|
| z } | j j | j | « « % y )NrÒ é r r )rX rö r Úcolumn_name_pointer_lengthÚ!column_name_text_subheader_offsetÚcolumn_name_offset_offsetÚcolumn_name_length_offsetr¼ Ú!column_name_text_subheader_lengthÚcolumn_name_offset_lengthÚcolumn_name_length_lengthre rf rû r× )rM rÏ rJ r Úcolumn_name_pointers_countrü Útext_subheaderÚcol_name_offsetÚcol_name_lengthÚidxÚ
|
col_offsetÚcol_lenÚname_rawÚcnames r& rv z,SAS7BDATReader._process_columnname_subheader" s` Ø×"Ñ"Ø'ÑØ&,¨q°7©{Ñ&:¸RÑ&?ÀAÑ%EÐ"ÜÐ1Ó2ó GAàÜ×2Ñ2°a¸!±eÑ<ñ=ä×9Ñ9ñ:ð ð Ü×2Ñ2°a¸!±eÑ<ñ=ä×1Ñ1ñ2ð ð Ü×2Ñ2°a¸!±eÑ<ñ=ä×1Ñ1ñ2ð ð //Ø ¤× GÑ GóCð Ø ¤×!@Ñ!@óJð oo o´u×7VÑ7VÓWGà×,Ñ,¨SÑ1HØ Z¨*°wÑ*>Ð?EØ × Ñ × $Ñ $ T×%>Ñ%>¸uÓ%EÖ Fñ7 Gr( c ó| | j }|d|z z
|
dz
|
|dz z }t |« D ] }||z t j z ||dz z z }|d|z z t j z ||dz z z }|d|z z t j
|
z ||dz z z }| j ||« } | j j | « | j |t j « } | j j | « | j |t j « } | j j | dk( rdnd« y )NrÒ r, r r ó dó s) rX rö r Úcolumn_data_offset_offsetÚcolumn_data_length_offsetÚcolumn_type_offsetr¼ rk rû Úcolumn_data_length_lengthrj Úcolumn_type_lengthrl )
|
rM rÏ rJ r Ú column_attributes_vectors_countrü Úcol_data_offsetÚ col_data_lenÚ col_typesrÆ s
|
r& rw z2SAS7BDATReader._process_columnattributes_subheaderC sO Ø×"Ñ"Ø+1°A¸±KÑ+?À"Ñ+DÈ'ÐTUÉ+Ñ*VÐ'ÜÐ6Ó7ó @AàÑ ¤5×#BÑ#BÑBÀQÈ'ÐTUÉ+ÑEVÑVð ð Øg+ñ ä×1Ñ1ñ2ð w {Ñ#ñ$ð ð WÑ$¤u×'?Ñ'?Ñ?À!ÀwÐQRÁ{ÑBSÑSð ð °Ó9AØ × %Ñ %× ,Ñ ,¨QÔ /à ¬e×.MÑ.MÓNAØ × %Ñ %× ,Ñ ,¨QÔ /à ¬5×+CÑ+CÓDAØ × Ñ × %Ñ %¨a°1ªf¡d¸$Ö ?ñ+ @r( c ó y rL rU rÔ s r& ry z,SAS7BDATReader._process_columnlist_subheader] s à r( c óì | j }|t j z d|z z }|t j z d|z z }|t j z d|z z }|t j
|
z d|z z }|t j z d|z z }|t j z d|z z } | j |t j « }
|
t |
|
t | j « dz
|
« } | j |t j « } | j |t j « } | j |t j « }t |t | j « dz
|
« }| j |t j « }| j | t j" « }| j | }| j% ||||z « }| j | }| j% || | | z « }t | j& « }t) || j* | ||| j, | | j. | « }| j0 j3 |« | j& j3 |« y )Né r )rX r Ú)column_format_text_subheader_index_offsetÚcolumn_format_offset_offsetÚcolumn_format_length_offsetÚ(column_label_text_subheader_index_offsetÚcolumn_label_offset_offsetÚcolumn_label_length_offsetr¼ Ú)column_format_text_subheader_index_lengthÚminr re Úcolumn_format_offset_lengthÚcolumn_format_length_lengthÚ(column_label_text_subheader_index_lengthÚcolumn_label_offset_lengthÚcolumn_label_length_lengthr× rh rA rf rl rj rg rû )rM rÏ rJ r Útext_subheader_formatÚcol_format_offsetÚcol_format_lenÚtext_subheader_labelÚcol_label_offsetÚ col_label_lenrÆ Ú
|
format_idxÚ format_startÚ
|
format_lenÚ label_idxÚ label_startÚ label_lenÚ label_namesÚ column_labelÚ format_namesÚ column_formatÚcurrent_column_numberÚcols r& rx z(SAS7BDATReader._process_format_subheadera sX Ø×"Ñ"à U×DÑDÑ DÀqÈ7Á{Ñ Rð ð #¤U×%FÑ%FÑ FÈÈWÉÑ TÐØ ¤%×"CÑ"CÑCÀaÈ'ÁkÑQà U×CÑCÑ CÀaÈ'ÁkÑ Qð ð "¤E×$DÑ$DÑDÀqÈ7Á{ÑRÐØ ¤×!AÑ!AÑAÀAÈÁKÑO à OOØ !¤5×#RÑ#Ró
|
ô C × 5Ñ 5Ó 6¸Ñ :Ó;
|
àØ u× @Ñ @ó
|
ð __ ^´U×5VÑ5VÓW
|
àOOØ ¤%×"PÑ"Pó
|
ô ¤3 t×'<Ñ'<Ó#=ÀÑ#AÓB àooØ e× >Ñ >ó
|
ð OO M´5×3SÑ3SÓT à×+Ñ+¨IÑ6 Ø×0Ñ0Ø k°IÑ&=Ð >ó
|
ð ×,Ñ,¨ZÑ8 Ø ×1Ñ1Ø ¨ °zÑ(AÐ Bó
|
ô !$ D§L¡LÓ 1Ð äØ !Ø × Ñ Ð 3Ñ 4Ø Ø Ø × Ñ Ð 4Ñ 5Ø × %Ñ %Ð&;Ñ <ó
|
ð ×Ñ×"Ñ" =Ô1Ø ×ÑCÕ r( c ó || j | j }n| | j }t | j « dk( r| j « t d« |dkD r#| j | j k\ r
|
t « S t || j | j z
|
« }| j j d« }| j j d« }t j ||ft ¬« | _ t j |d|z ft j ¬« | _ d| _ t% | « }|j' |« | j) « }| j* |j- | j* « }|S )Nr z No columns to parse from filer> r? r/ r )r` r r rl r~ r rm r rS Úcountr8 rÉ ÚobjectÚ _string_chunkÚzerosÚuint8Ú _byte_chunkÚ_current_row_in_chunk_indexr r Ú_chunk_to_dataframer- Ú set_index)rM rÈ ÚndÚnsÚpÚrslts r& r zSAS7BDATReader.read s7 Ø M §¡Ð :ØNNEØ ]ØNNEä t×!Ñ!Ó " aÒ 'Ø JJLÜ Ð!@ÓAÐ Aà 19×8Ñ8¸D¿N¹NÒJÜ ;Ð äE4>>¨D×,KÑ,KÑKÓLà × Ñ × %Ñ % dÓ +Ø × Ñ × %Ñ % dÓ +ä XX r¨5 k¼Ô @ÔÜ 88 R¨¨U© O¼2¿8¹8ÔDÔà+,Ô(Ü 4LØ u à×'Ñ'Ó)Ø ::Ð !Ø>> $§*¡*Ó-Dà r( c ót g | _ | j j | j « | _ t | j « dk ryt | j « | j k7 rC| j « dt | j « dd| j dd}t |« | j « | j t j v r| j « | j t j t j t j gz vr| j « S y)Nr Tz-failed to read complete page from file (read r z of z bytes)F)ri rq r r rY r r~ r# rÞ râ r rß rã rä rá Ú_read_next_page)rM Úmsgs r& rz z SAS7BDATReader._read_next_page· s Ø57Ô2Ø × -Ñ -× 2Ñ 2°4×3DÑ3DÓ EÔÜ t× Ñ Ó ! QÒ &ØÜ ×"Ñ"Ó # t×'8Ñ'8Ò 8Ø JJLà?Üt×(Ñ(Ó)¨!Ð,¨D°×1BÑ1BÀ1Ð0EÀWðNð ô S/Ð !à × Ñ Ô Ø × "Ñ "¤e×&;Ñ&;Ñ ;Ø × 'Ñ 'Ô )à × "Ñ "¬%×*?Ñ*?Ü × Ñ Ü × Ñ ðC
|
ñ +
|
ñ
|
ð ×'Ñ'Ó)Ð )àr( c óT | j }| j }t ||z
|
|« }i }d\ }}t d« }t | j « D ]¼ }| j
|
| } | j | dk( rÏ| j |d d
f j | j dz ¬« }
|
t j |
|
t j |d¬« || < | j rc| j | t j" v rt% || d« || < n1| j | t j& v rt% || d« || < |d z }ô| j | d
|
k( r
t j | j( |d d
f |d¬ « || < | j* rF| j, :| j/ || j0 « || < |r|| j3 d « || < |d z }| j5 « t7 d t9 | j | « « t; || j
|
|d¬« } | S )N)r r zfuture.infer_stringr> r r/ Fr1 r r r? r, Ústrzunknown column type )rh r- r. ) rr rm rö r r rf rl rq r6 r¬ r! r7 r8 Úfloat64r^ rg r Úsas_date_formatsr? Úsas_datetime_formatsrn rb ra Ú_decode_stringr} Úastyper~ r# Úreprr ) rM ÚnÚmÚixrx ÚjsÚjbÚ infer_stringÚjrE Úcol_arrÚdfs r& rs z"SAS7BDATReader._chunk_to_dataframeÐ s Ø × ,Ñ ,Ø × +Ñ +Ü 1q5!_ØàBÜ!Ð"7Ó8 Üt×(Ñ(Ó)ó WAØ×$Ñ$ QÑ'Dà×!Ñ! !Ñ$¨Ò,Ø ×*Ñ*¨2ªq¨5Ñ1×6Ñ6¸T¿_¹_ÈsÑ=RÐ6ÓSÜ YY w´b·j±jÈÐQVÔ WT
|
Ø×%Ò%Ø×*Ñ*¨1Ñ-´×1GÑ1GÑGÜ%7¸¸T¹
|
ÀCÓ%HT
|
Ø ×,Ñ,¨QÑ/´5×3MÑ3MÑMÜ%7¸¸T¹
|
ÀCÓ%HT
|
ØaØ×#Ñ# AÑ&¨$Ò.Ü YY t×'9Ñ'9¸"ºa¸%Ñ'@ÈÐQVÔ WT
|
Ø×$Ò$¨$¯-©-Ð*CØ!%×!4Ñ!4°T¸$±Z·^±^Ó!DDJÙ#Ø%)¨$¡Z×%6Ñ%6°uÓ%=T
|
àaà
|
|
Ü Ð#7¼¸T×=OÑ=OÐPQÑ=RÓ8SÐ7TÐ!UÓVÐVð- Wô0 t T×%6Ñ%6¸bÀuÔ MØ r( c óT |j | j xs | j « S rL )Údecodera rd ©rM Úbs r& r z SAS7BDATReader._decode_stringó s Øxx Ò>¨×)>Ñ)>Ó?Ð?r( c ó@ | j r| j |« S |S rL )rc r r s r& r× z#SAS7BDATReader._convert_header_textö s! Ø × #Ò #Ø×&Ñ& qÓ)Ð )àHr( )NTTNNTTr )r z FilePath | ReadBuffer[bytes]r^ rå r_ rå r` ú
|
int | Nonera z
|
str | Nonerb rå rc rå r] r rO rP )rO z
|
np.ndarray)rO rP )rO r )rÏ rB rÐ rB )rÏ rB rÐ rB rO rB )rÏ rB rJ rB )rÏ rB rJ rB rO rD )rO rå )rÏ rB rJ rB rO rP rL )rÈ r rO r )r rH rO rD )!rQ rR rS Ú__doc__rT rN r r r r~ r{ rÌ r´ r¼ r rÙ r| rÛ rÞ rã rr rs rt ru rv rw ry rx r rz rs r r× rU r( r& rW rW s#
ñ ð< ÓØ Ó ð
|
Ø "Ø "Ø $Ø #Ø !Ø$(Ø*1ð=à1ð=ð ð =ð
|
ð =ð ð =ð ð=ð ð=ð "ð=ð (ð=ð
|
ó=ó~EóEóDó óD
|
óLó 4ó 2ó(;ó
|
ó
|
-ó
|
ó
|
ó)HóV 3ó<ó ó+QóZ GóB@ó4 ó5!ôn ò>ó2!òF@ôr( rW )r$ Úfloatr% r} )r: ú pd.Seriesr% r} rO r )4r Ú
|
__future__r Ú collectionsr r r r Útypingr Únumpyr8 Úpandas._configr Úpandas._libs.byteswapr r
|
r r r Úpandas._libs.sasr r Ú pandas._libs.tslibs.conversionr Ú pandas.errorsr Úpandasr! r r r Úpandas.io.commonr Úpandas.io.sas.sas_constantsÚioÚsasÚ sas_constantsr Úpandas.io.sas.sasreaderr Úpandas._typingr r r r3 r2 r' r? rA ÚIteratorrW rU r( r& ú<module>r¨ s¤ ðñõ #å ÷ó Ý ã å %÷õ ÷õ EÝ (ã ÷ñ õ (ß +Ó +Ý .á÷ñ ñ Ó& Ù Ó% ó 4ó U÷< ñ ô6p Z § ¡ õ p r(
|
