Ë
|
nñúh<C ã ó d dl mZ d dlmZmZ d dlZd dlmZmZm Z d dl
|
Z d dl m Z d dlZd dlmZ erd dlmZ d dlmZmZ dd Z d dd
|
Z dd Zdd Z d dd Z d ddZy)é )Ú annotations)ÚabcÚ defaultdictN)Ú TYPE_CHECKINGÚAnyÚ DefaultDict©Úconvert_json_to_lines)Ú DataFrame)ÚIterable)Ú IgnoreRaiseÚScalarc óF | d dk( s
|
| d dk( r| S | dd } t | « S )zJ
|
Helper function that converts JSON lists to line delimited JSON.
|
r ú[éÿÿÿÿú]é r )Úss úLH:\Change_password\venv_build\Lib\site-packages\pandas/io/json/_normalize.pyÚconvert_to_line_delimitsr s4 ð Q43;1R5 C<ØØ !BAä Ó #Ð#ó c óð d}t | t « r| g} d}g }| D ]Ð }t j |« }|j « D ] \ } }
|
t | t
|
« s t | « } |dk( r| } n||z | z } t |
|
t « r|!||k\ r |dk7 r|j | « }
|
|
|
|| < d|j | « }
|
|j t |
|
| ||dz |« « |j |« Ò |r|d S |S )a
|
A simplified json_normalize
|
|
Converts a nested dict into a flat dict ("record"), unlike json_normalize,
|
it does not attempt to extract a subset of the data.
|
|
Parameters
|
----------
|
ds : dict or list of dicts
|
prefix: the prefix, optional, default: ""
|
sep : str, default '.'
|
Nested records will generate names separated by sep,
|
e.g., for sep='.', { 'foo' : { 'bar' : 0 } } -> foo.bar
|
level: int, optional, default: 0
|
The number of levels in the json string.
|
|
max_level: int, optional, default: None
|
The max depth to normalize.
|
|
Returns
|
-------
|
d - dict or list of dicts, matching `ds`
|
|
Examples
|
--------
|
>>> nested_to_record(
|
... dict(flat1=1, dict1=dict(c=1, d=2), nested=dict(e=dict(c=1, d=2), d=2))
|
... )
|
{'flat1': 1, 'dict1.c': 1, 'dict1.d': 2, 'nested.e.c': 1, 'nested.e.d': 2, 'nested.d': 2}
|
FTr r )
|
Ú
|
isinstanceÚdictÚcopyÚdeepcopyÚitemsÚstrÚpopÚupdateÚnested_to_recordÚappend) ÚdsÚprefixÚsepÚlevelÚ max_levelÚ singletonÚnew_dsÚdÚnew_dÚkÚvÚnewkeys r r! r! - s ðX IÜ"dÔØTØ Ø FØ ò Ü aÓ ØGGIò QDAqä a¤Ô%ÜFØzØà # ¨Ñ)ô a¤Ô&ØÐ%¨%°9Ò*<àA:Ø ! AØ$%E&MØà ! AØ LLÔ)¨!¨V°S¸%À!¹)ÀYÓOÕ Pð- Qð. eÕ ð3 ñ6 ØayÐØ Mr c ó¸ t | t « rD| j « D ]/ \ }}| | | }|s|j |« }t ||||¬« 1 |S | ||< |S )a3
|
Main recursive function
|
Designed for the most basic use case of pd.json_normalize(data)
|
intended as a performance improvement, see #15621
|
|
Parameters
|
----------
|
data : Any
|
Type dependent on types contained within nested Json
|
key_string : str
|
New key (with separator(s) in) for data
|
normalized_dict : dict
|
The new normalized/flattened Json dict
|
separator : str, default '.'
|
Nested records will generate names separated by sep,
|
e.g., for sep='.', { 'foo' : { 'bar' : 0 } } -> foo.bar
|
©ÚdataÚ
|
key_stringÚnormalized_dictÚ separator)r r r Ú removeprefixÚ_normalise_json)r1 r2 r3 r4 ÚkeyÚvalueÚnew_keys r r6 r6 ~ sz ô. $Ô Ø **,ò JCØ# Y K°¨uÐ5Gá Ø!×.Ñ.¨yÓ9ä ØØ"Ø /Ø#ö ð ð Ðð '+
|
Ñ#Ø Ðr c
|
ó | j « D ci c] \ }}t |t « r|| }}}t | j « D ci c] \ }}t |t « s|| c}}di |¬« }i |¥|¥S c c}}w c c}}w )aw
|
Order the top level keys and then recursively go to depth
|
|
Parameters
|
----------
|
data : dict or list of dicts
|
separator : str, default '.'
|
Nested records will generate names separated by sep,
|
e.g., for sep='.', { 'foo' : { 'bar' : 0 } } -> foo.bar
|
|
Returns
|
-------
|
dict or list of dicts, matching `normalised_json_object`
|
Ú r0 )r r r r6 )r1 r4 r, r- Ú top_dict_Ú nested_dict_s r Ú_normalise_json_orderedr> § s ð #'§*¡*£,×J$!Q´jÀÄDÕ6IAÐJIÑJÜ"Ø #zz|× Ctq!¬z¸!¼TÕ/BadÓ CØØØô Lð )iÐ (<Ð (Ð(ùó Kùã Cs A=A=Á B
|
Á&B
|
c óª i }t | t « rt | |¬« }|S t | t « r | D cg c] }t ||¬« }}|S |S c c}w )a
|
A optimized basic json_normalize
|
|
Converts a nested dict into a flat dict ("record"), unlike
|
json_normalize and nested_to_record it doesn't do anything clever.
|
But for the most basic use cases it enhances performance.
|
E.g. pd.json_normalize(data)
|
|
Parameters
|
----------
|
ds : dict or list of dicts
|
sep : str, default '.'
|
Nested records will generate names separated by sep,
|
e.g., for sep='.', { 'foo' : { 'bar' : 0 } } -> foo.bar
|
|
Returns
|
-------
|
frame : DataFrame
|
d - dict or list of dicts, matching `normalised_json_object`
|
|
Examples
|
--------
|
>>> _simple_json_normalize(
|
... {
|
... "flat1": 1,
|
... "dict1": {"c": 1, "d": 2},
|
... "nested": {"e": {"c": 1, "d": 2}, "d": 2},
|
... }
|
... )
|
{'flat1': 1, 'dict1.c': 1, 'dict1.d': 2, 'nested.e.c': 1, 'nested.e.d': 2, 'nested.d': 2}
|
|
)r1 r4 ©r% )r r r> ÚlistÚ_simple_json_normalize)r# r% Únormalised_json_objectÚrowÚnormalised_json_lists r rB rB À sd ðV Ðä"dÔÜ!8¸bÈCÔ!PÐ ð "Ð!ô
|
BÔ ØPRÖ SÈÔ 6°sÀÖ DÐ SÐ Ð SØ#Ð#Ø !Ð!ùò Ts ¶Ac ó¤ d dfd dfd t | t « r | s
|
t « S t | t « r| g} n<t | t j
|
« r t | t « s t | « } nt | | |t t | ¬« « S |+t d | D « « rt | ¬« } t | « S t |t « s|g}|g }nt |t « s|g}|D cg c] }t |t « r|n|g c}g g t t « D cg c] } j | « c} ddf
|
d | |i d¬« t « }
|
|
|
j fd ¬
|
« }
|
j « D ] \ } } ||| z } | |
|
v rt d | d « t! j" | t$ ¬ « } | j& dkD r=t! j( t+ | « ft$ ¬ « } t- | « D ]
|
\ }} | | |< | j/ « |
|
| < |
|
S c c}w c c} w )a´
|
Normalize semi-structured JSON data into a flat table.
|
|
Parameters
|
----------
|
data : dict or list of dicts
|
Unserialized JSON objects.
|
record_path : str or list of str, default None
|
Path in each object to list of records. If not passed, data will be
|
assumed to be an array of records.
|
meta : list of paths (str or list of str), default None
|
Fields to use as metadata for each record in resulting table.
|
meta_prefix : str, default None
|
If True, prefix records with dotted (?) path, e.g. foo.bar.field if
|
meta is ['foo', 'bar'].
|
record_prefix : str, default None
|
If True, prefix records with dotted (?) path, e.g. foo.bar.field if
|
path to records is ['foo', 'bar'].
|
errors : {'raise', 'ignore'}, default 'raise'
|
Configures error handling.
|
|
* 'ignore' : will ignore KeyError if keys listed in meta are not
|
always present.
|
* 'raise' : will raise KeyError if keys listed in meta are not
|
always present.
|
sep : str, default '.'
|
Nested records will generate names separated by sep.
|
e.g., for sep='.', {'foo': {'bar': 0}} -> foo.bar.
|
max_level : int, default None
|
Max number of levels(depth of dict) to normalize.
|
if None, normalizes all levels.
|
|
Returns
|
-------
|
frame : DataFrame
|
Normalize semi-structured JSON data into a flat table.
|
|
Examples
|
--------
|
>>> data = [
|
... {"id": 1, "name": {"first": "Coleen", "last": "Volk"}},
|
... {"name": {"given": "Mark", "family": "Regner"}},
|
... {"id": 2, "name": "Faye Raker"},
|
... ]
|
>>> pd.json_normalize(data)
|
id name.first name.last name.given name.family name
|
0 1.0 Coleen Volk NaN NaN NaN
|
1 NaN NaN NaN Mark Regner NaN
|
2 2.0 NaN NaN NaN NaN Faye Raker
|
|
>>> data = [
|
... {
|
... "id": 1,
|
... "name": "Cole Volk",
|
... "fitness": {"height": 130, "weight": 60},
|
... },
|
... {"name": "Mark Reg", "fitness": {"height": 130, "weight": 60}},
|
... {
|
... "id": 2,
|
... "name": "Faye Raker",
|
... "fitness": {"height": 130, "weight": 60},
|
... },
|
... ]
|
>>> pd.json_normalize(data, max_level=0)
|
id name fitness
|
0 1.0 Cole Volk {'height': 130, 'weight': 60}
|
1 NaN Mark Reg {'height': 130, 'weight': 60}
|
2 2.0 Faye Raker {'height': 130, 'weight': 60}
|
|
Normalizes nested data up to level 1.
|
|
>>> data = [
|
... {
|
... "id": 1,
|
... "name": "Cole Volk",
|
... "fitness": {"height": 130, "weight": 60},
|
... },
|
... {"name": "Mark Reg", "fitness": {"height": 130, "weight": 60}},
|
... {
|
... "id": 2,
|
... "name": "Faye Raker",
|
... "fitness": {"height": 130, "weight": 60},
|
... },
|
... ]
|
>>> pd.json_normalize(data, max_level=1)
|
id name fitness.height fitness.weight
|
0 1.0 Cole Volk 130 60
|
1 NaN Mark Reg 130 60
|
2 2.0 Faye Raker 130 60
|
|
>>> data = [
|
... {
|
... "state": "Florida",
|
... "shortname": "FL",
|
... "info": {"governor": "Rick Scott"},
|
... "counties": [
|
... {"name": "Dade", "population": 12345},
|
... {"name": "Broward", "population": 40000},
|
... {"name": "Palm Beach", "population": 60000},
|
... ],
|
... },
|
... {
|
... "state": "Ohio",
|
... "shortname": "OH",
|
... "info": {"governor": "John Kasich"},
|
... "counties": [
|
... {"name": "Summit", "population": 1234},
|
... {"name": "Cuyahoga", "population": 1337},
|
... ],
|
... },
|
... ]
|
>>> result = pd.json_normalize(
|
... data, "counties", ["state", "shortname", ["info", "governor"]]
|
... )
|
>>> result
|
name population state shortname info.governor
|
0 Dade 12345 Florida FL Rick Scott
|
1 Broward 40000 Florida FL Rick Scott
|
2 Palm Beach 60000 Florida FL Rick Scott
|
3 Summit 1234 Ohio OH John Kasich
|
4 Cuyahoga 1337 Ohio OH John Kasich
|
|
>>> data = {"A": [1, 2]}
|
>>> pd.json_normalize(data, "A", record_prefix="Prefix.")
|
Prefix.0
|
0 1
|
1 2
|
|
Returns normalized data with columns prefixed with the given string.
|
c ó | } t |t « r |D ] }| t |« || } |S || } |S # t $ rD}|rt d| d« |dk( rt j cY d}~S t d| d| d« |d}~ww xY w)z Internal function to pull fieldNzKey zS not found. If specifying a record_path, all elements of data should have the path.Úignorez) not found. To replace missing values of z% with np.nan, pass in errors='ignore')r rA ÚKeyErrorÚnpÚnan)ÚjsÚspecÚextract_recordÚresultÚfieldÚeÚerrorss r Ú _pull_fieldz#json_normalize.<locals>._pull_field sÏ ø ð ð Ü$¤Ô%Ø !ò +EØ ~Ü & uoÐ-Ø # E]Fñ+ð( ð ð øô ò Ù Ü Ø1#ð 1ð 2óð ðð Ò!Üvv ä Ø1#Ð FÀqÀcð J6ð 7óð ðûð ús%
)9 ±9 ¹ BÁ&BÁ(BÁ.BÂBc ó | |d¬« }t |t « s-t j |« rg }|S t | d| d| d« |S )z¶
|
Internal function to pull field for records, and similar to
|
_pull_field, but require to return list. And will raise error
|
if has non iterable value.
|
T)rN z has non list value z
|
for path z. Must be list or null.)r rA ÚpdÚisnullÚ TypeError)rL rM rO rS s r Ú _pull_recordsz%json_normalize.<locals>._pull_records sj ø ñ R °dÔ;ô &¤$Ô'ÜyyÔ Øð ô ØdÐ.¨v¨h°jÀÀð G,ð ,óð ð r r@ c 3 ó~ K | ]0 }|j « D cg c] }t |t « c} 2 y c c}w w©N)Úvaluesr r )Ú.0ÚyÚxs r ú <genexpr>z!json_normalize.<locals>.<genexpr>Ì s+ è ø ÒG¸Q¨Q¯X©X«ZÖ8¨
|
1dÕ#Ö8ÑGùÒ8ùs =8° =©r% r' r c óh
|
t | t « r| g} t |« dkD rU| D ]O }t
|
« D ]& \ }}|dz t |« k( s ||d « ||< ( ||d |dd ||dz ¬« Q y | D ]± } ||d « }|D cg c]" }t |t « rt |¬« n|$ }}j t |« « t
|
« D ]< \ }}|dz t |« kD r|| } n |||d « } | j | « > j |« ³ y c c}w )Nr r r ©r& r` )r r ÚlenÚzipr! r" Úextend)r1 ÚpathÚ seen_metar& ÚobjÚvalr7 ÚrecsÚrÚmeta_valÚ_metarS rX Ú_recursive_extractÚlengthsr' Ú meta_keysÚ meta_valsÚrecordsr% s r rn z*json_normalize.<locals>._recursive_extractç sn ø Ü dDÔ !Ø6DÜ t9q=Øò WÜ # E¨9Ó 5ò CHCØ qy¤C¨£HÓ,Ù)4°S¸#¸b¹'Ó)B #ðCñ # 3 t¨A¡w¡<°°a°b°¸9ÈEÐTUÉIÖVñ Wð ò %Ù$ S¨$¨q©'Ó2ð
|
"ö ð ô " !¤TÔ*ô % Q¨C¸9ÕEàñðð ð s 4yÔ)Ü # E¨9Ó 5ò 4HCØ qy¤3 s£8Ò+Ø#,¨S¡>á#.¨s°C¸¸°KÓ#@Ø cN×)Ñ)¨(Õ3ð 4ð tÕ$ñ# %ùòs  'D/rb c ó | S rZ © )r^ Ú record_prefixs r ú<lambda>z json_normalize.<locals>.<lambda>
|
s ø °M°?À1À#Ð1F r )ÚcolumnszConflicting metadata name z , need distinguishing prefix )Údtyper )F)rL údict[str, Any]rM ú
|
list | strrN ÚboolÚreturnzScalar | Iterable)rL ry rM rz r| rA )r )r& Úintr| ÚNone)r rA r r r r r ÚNotImplementedErrorrB Úanyr! r ÚjoinÚrenamer Ú
|
ValueErrorrJ ÚarrayÚobjectÚndimÚemptyrc Ú enumerateÚrepeat)r1 Ú record_pathÚmetaÚ meta_prefixru rR r% r' Úmri rO r, r- r[ Úirm rS rX rn ro rp rq rr s ```` @@@@@@@@r Újson_normalizer õ s_ ÿû ð\ FKðØ ðØ",ðØ>Bðà õõ:ô( $Ô ¡dÜ{ÐÜ D$Ô àvÜ D#,,Ô '´
|
¸4ÄÔ0Eô Dzä!Ð!ð ÐØ LØ Ð Ø Ð !Ø Ð äÔ/°¸#Ô>Ó?Ð?àÐÜ ÑGÀ$ÔGÔ Gô $ D¨c¸YÔGDÜÐ Ü ¤TÔ *Ø"m à |ØÜ dÔ #Øvà8<Ö =°1*Q¤Ô%Q¨A¨3Ñ .Ò =Eð GØGä (¬Ó .IØ*/Ö0 3# Ò0I÷ %÷ %ñ< t[¨"°AÕ6ä wÓ FàÐ Ø Ó'F ÓGð Ó!ò +1Ø Ð "ØaAà ;Ü Ø,¨Q¨CÐ/LÐMóð ô
|
!¤6Ô*à ;;?äXXs 1vi¬vÔ6FÜ! ! ò 1Ø q ð ð MM 'Ó*q ð%+ð& MùòE >ùò 1s Ä
|
IÅ I )r r r| r )r; ú.r N)r$ r r% r r& r} r' ú
|
int | None)
|
r1 r r2 r r3 ry r4 r r| ry )r1 ry r4 r r| ry )r )r# údict | list[dict]r% r r| zdict | list[dict] | Any)NNNNÚraiser N)r1 r r zstr | list | Noner z"str | list[str | list[str]] | Noner ú
|
str | Noneru r rR r r% r r' r r| r ) Ú
|
__future__r Ú collectionsr r r Útypingr r r ÚnumpyrJ Úpandas._libs.writersr
|
ÚpandasrU r Úcollections.abcr Úpandas._typingr r r r! r6 r> rB r rt r r ú<module>r sU ðõ #÷ó ÷ñ ó å 6ã Ý áÝ(÷ó
|
$ð ØØØ ð Nà ðNð
|
ðNð ð Nð
|
ó Nðb&Ø
|
ð&àð&ð $ð&ð ð &ð
|
ó &óR)ð6 ð2"Øð2"à ð2"ð ó2"ðn &*Ø/3Ø "Ø $Ø!ØØ ðkØ
|
ðkà"ðkð -ðkð ð kð
|
ð kð ð kð
|
ðkð ðkð ôkr
|
