Ë
|
ñúh9? ã ó² d Z ddlmZ ddlZddlZddlZddlZddlZddl Z ddl
|
Z
|
ddl Z ddl Z ddlmZmZmZ ddlmZ ddlmZ ddlmZ ddlmZmZmZ dd l m Z dd
|
l m Z dd l m!Z!m"Z" dd l#m$Z$ dd l%m&Z& ddl'm(Z( ddl)m*Z* ddl+m,Z, ddl-m.Z. ddl/m0Z0 ddl1m2Z2 ddl3m4Z4m5Z5m6Z6 e jn e8« Z9ee:e:f Z;d2dZ< G d de=« Z>d3dZ? G d de=« Z@d4d ZAd5d ZBd6d ZC G d d « ZD G d! d"e« ZEd7d#ZFeFd8d%« ZG ed&¬'« G d( d$« « ZH G d) d*e« ZI d9 d:d+ZJ d; d<d,ZKd=d-ZL G d. d/e« ZM G d0 d1« ZNy)>zO
|
The main purpose of this module is to expose LinkCollector.collect_sources().
|
é )Ú annotationsN)ÚIterableÚMutableMappingÚSequence)Ú dataclass)Ú
|
HTMLParser)ÚValues)ÚCallableÚ
|
NamedTupleÚProtocol)Úrequests)ÚResponse)Ú
|
RetryErrorÚSSLError)ÚNetworkConnectionError)ÚLink)Ú SearchScope)Ú
|
PipSession)Úraise_for_status)Úis_archive_file©Úredact_auth_from_url)Úvcsé )ÚCandidatesFromPageÚ
|
LinkSourceÚ build_sourcec ó t j D ]6 }| j « j |« s#| t |« dv s4|c S y)zgLook for VCS schemes in the URL.
|
|
Returns the matched VCS scheme, or None if there's no match.
|
z+:N)r ÚschemesÚlowerÚ
|
startswithÚlen)ÚurlÚschemes úPH:\Change_password\venv_build\Lib\site-packages\pip/_internal/index/collector.pyÚ_match_vcs_schemer& . sE ô
|
++ò Ø 99;× !Ñ ! &Õ )¨c´#°f³+Ñ.>À$Ò.FØMðð ó c ó e Zd Zd fd Z xZS )Ú_NotAPIContentc óB t | ||« || _ || _ y ©N)ÚsuperÚ__init__Ú content_typeÚ request_desc)Úselfr. r/ Ú __class__s r% r- z_NotAPIContent.__init__: s" ø Ü Ñ |Ô4Ø (ÔØ (Õr' )r. Ústrr/ r2 ÚreturnÚNone)Ú__name__Ú
|
__module__Ú __qualname__r- Ú __classcell__©r1 s @r% r) r) 9 s ø ÷)ñ )r' r) c ó¾ | j j dd« }|j « }|j d« ryt || j
|
j « )z°
|
Check the Content-Type header to ensure the response contains a Simple
|
API Response.
|
|
Raises `_NotAPIContent` if the content type is not a valid content-type.
|
ú Content-TypeÚUnknown)z text/htmlz#application/vnd.pypi.simple.v1+htmlú#application/vnd.pypi.simple.v1+jsonN)ÚheadersÚgetr r! r) ÚrequestÚmethod)Úresponser. Úcontent_type_ls r% Ú_ensure_api_headerrD @ s[ ð ×#Ñ#×'Ñ'¨¸ ÓBLà!×'Ñ'Ó)NØ× Ñ ð
|
ôð ä
|
x×'7Ñ'7×'>Ñ'>Ó
|
?Ð?r' c ó e Zd Zy)Ú_NotHTTPN)r5 r6 r7 © r' r% rF rF V s Ør' rF c ó¼ t j j | « \ }}}}}|dvr
|
t « |j | d¬« }t |« t |« y)zõ
|
Send a HEAD request to the URL, and ensure the response contains a simple
|
API Response.
|
|
Raises `_NotHTTP` if the URL is not available for a HEAD request, or
|
`_NotAPIContent` if the content type is not a valid content type.
|
> ÚhttpÚhttpsT)Úallow_redirectsN)ÚurllibÚparseÚurlsplitrF Úheadr rD )r# Úsessionr$ ÚnetlocÚpathÚqueryÚfragmentÚresps r% Ú_ensure_api_responserV Z sV ô -3¯L©L×,AÑ,AÀ#Ó,FÑ)FFD% Ø Ð&Ñ&ÜjÐà <<¨T<Ó 2DÜTÔätÕ r' c ó t t | « j « r t | |¬« t j dt | « « |j | dj g d¢« dd¬« }t |« t |« t j dt | « |j j d d
|
« « |S ) aY Access an Simple API response with GET, and return the response.
|
|
This consists of three parts:
|
|
1. If the URL looks suspiciously like an archive, send a HEAD first to
|
check the Content-Type is HTML or Simple API, to avoid downloading a
|
large file. Raise `_NotHTTP` if the content type cannot be determined, or
|
`_NotAPIContent` if it is not HTML or a Simple API.
|
2. Actually perform the request. Raise HTTP exceptions on network failures.
|
3. Check the Content-Type header to make sure we got a Simple API response,
|
and raise `_NotAPIContent` otherwise.
|
©rP zGetting page %sz, )r= z*application/vnd.pypi.simple.v1+html; q=0.1ztext/html; q=0.01z max-age=0)ÚAcceptz Cache-Control)r> zFetched page %s as %sr; r< ) r r ÚfilenamerV ÚloggerÚdebugr r? Újoinr rD r> )r# rP rU s r% Ú_get_simple_responser^ l s¬ ô tCy×)Ñ)Ô*Ü S¨'Õ2ä
|
LLÐ"Ô$8¸Ó$=Ô>à ;;Ø àiiòóð( )ñ+
|
ð ó Dô4 TÔô tÔ ä
|
LLØ Ü SÓ!Ø ×Ñ ¨Ó3ôð Kr' c ó | rHd| v rDt j j « }| d |d< |j d« }|r t |« S y)z=Determine if we have any encoding information in our headers.r; z content-typeÚcharsetN)ÚemailÚmessageÚMessageÚ get_paramr2 )r> Úmr` s r% Ú_get_encoding_from_headersrf « sK á> WÑ,Ü MM× !Ñ !Ó #Ø # NÑ 3.ÑØ++iÓ(Ù Üw<Ð Ø r' c ó$ e Zd ZddZddZddZy)ÚCacheablePageContentc ó. |j sJ || _ y r+ )Úcache_link_parsingÚpage©r0 rk s r% r- z CacheablePageContent.__init__· s Ø×&Ò&Ð&Ð&Ø r' c ó t |t | « « xr- | j j |j j k( S r+ )Ú
|
isinstanceÚtyperk r# )r0 Úothers r% Ú__eq__zCacheablePageContent.__eq__» s- Ü%¤ d£Ó,ÒP°·±· ± À%Ç*Á*Ç.Á.Ñ1PÐPr' c ó@ t | j j « S r+ )Úhashrk r# ©r0 s r% Ú__hash__z CacheablePageContent.__hash__¾ s ÜDIIMMÓ"Ð"r' N)rk Ú IndexContentr3 r4 )rp Úobjectr3 Úbool)r3 Úint)r5 r6 r7 r- rq ru rG r' r% rh rh ¶ s óóQô#r' rh c ó e Zd ZddZy)Ú
|
ParseLinksc ó y r+ rG rl s r% Ú__call__zParseLinks.__call__Ã s r' N©rk rv r3 zIterable[Link])r5 r6 r7 r} rG r' r% r{ r{ Â s ÜAr' r{ c óz t j d fd « t j « d fd « }|S )zÚ
|
Given a function that parses an Iterable[Link] from an IndexContent, cache the
|
function's result (keyed by CacheablePageContent), unless the IndexContent
|
`page` has `page.cache_link_parsing == False`.
|
c ó: t | j « « S r+ )Úlistrk )Úcacheable_pageÚfns r% Úwrapperz*with_cached_index_content.<locals>.wrapperÍ s ø äB~×*Ñ*Ó+Ó,Ð,r' c ó` | j r t | « « S t | « « S r+ )rj rh r )rk r r s r% Úwrapper_wrapperz2with_cached_index_content.<locals>.wrapper_wrapperÑ s+ ø à × "Ò "ÙÔ/°Ó5Ó6Ð 6ÜBtH~Ð r' )r rh r3 ú
|
list[Link])rk rv r3 r )Ú functoolsÚcacheÚwraps)r r r s ` @r% Úwith_cached_index_contentr Æ s@ ù ô __ô-ó ð-ô __RÓõ ó ð ð
|
Ðr' rv c # óT K | j j « }|j d« r^t j | j
|
« }|j dg « D ]) }t j || j « }|&| + yt | j « }| j xs d}|j | j
|
j |« « | j }|j xs |}|j D ]! } t j | ||¬« }| | # yw)z\
|
Parse a Simple API's Index Content, and yield its anchor elements as Link objects.
|
r= ÚfilesNzutf-8)Úpage_urlÚbase_url)r. r r! ÚjsonÚloadsÚcontentr? r Ú from_jsonr# ÚHTMLLinkParserÚencodingÚfeedÚdecoder ÚanchorsÚ from_element)
|
rk rC ÚdataÚfileÚlinkÚparserr r# r Úanchors
|
r% Ú parse_linksr Ú sÿ è ø ð ×&Ñ&×,Ñ,Ó.NØ× Ñ Ð!FÔGÜzz$,,Ó'ØHHW bÓ)ò DÜ>> $¨¯©Ó1DØ|ØØJð ð
|
ä DHHÓ %FØ}}Ò' HØ
|
KK ×#Ñ# HÓ-Ô.à
|
((CØÒ% #HØ..ò Ü× Ñ °#ÀÔIØ <Ø Ø
|
ñ ùs D&D(T)Úfrozenc óP e Zd ZU dZded< ded< ded< ded< d Zd
|
ed < dd Zy )rv a Represents one response (or page), along with its URL.
|
|
:param encoding: the encoding to decode the given content.
|
:param url: the URL from which the HTML was downloaded.
|
:param cache_link_parsing: whether links parsed from this page's url
|
should be cached. PyPI index urls should
|
have this set to False, for example.
|
Úbytesr r2 r. ú
|
str | Noner r# Trx rj c ó, t | j « S r+ )r r# rt s r% Ú__str__zIndexContent.__str__ s Ü# D§H¡HÓ-Ð-r' N)r3 r2 )r5 r6 r7 Ú__doc__Ú__annotations__rj r¥ rG r' r% rv rv ÷ s/
ñð NØÓØÓØ HØ #ÐÓ#ô.r' c ó4 e Zd ZdZd fd ZddZddZ xZS )r zf
|
HTMLParser that keeps the first base HREF and a list of all anchor
|
elements' attributes.
|
c óP t | d¬« || _ d | _ g | _ y )NT)Úconvert_charrefs)r, r- r# r r )r0 r# r1 s r% r- zHTMLLinkParser.__init__ s( ø Ü Ñ¨$ÐÔ/àØ$( Ø46 r' c ó² |dk( r(| j | j |« }||| _ y y |dk( r%| j j t |« « y y )NÚbaseÚa)r Úget_hrefr ÚappendÚdict)r0 ÚtagÚattrsÚhrefs r% Úhandle_starttagz HTMLLinkParser.handle_starttag sT Ø &=T]]Ð 2Ø== Ó'DØÐ Ø $ ð à CZØ LL× Ñ ¤ U£ Õ ,ð r' c ó* |D ] \ }}|dk( s |c S y )Nr³ rG )r0 r² ÚnameÚvalues r% r® zHTMLLinkParser.get_href! s& Ø ò KD%Øv~Ø ð ð r' )r# r2 r3 r4 )r± r2 r² ú list[tuple[str, str | None]]r3 r4 )r² r¸ r3 r£ )r5 r6 r7 r¦ r- r´ r® r8 r9 s @r% r r s ø ñõ
|
7ó-÷r' r c ó< |t j } |d| |« y )Nz%Could not fetch URL %s: %s - skipping)r[ r\ )r ÚreasonÚmeths r% Ú_handle_get_simple_failr¼ ( s ð
|
|Ü||ÙÐ 0°$¸Õ?r' c ó t | j « }t | j | j d || j |¬« S )Nr; )r r# rj )rf r> rv r r# )rB rj r s r% Ú_make_index_contentr¾ 2 sE ô *¨(×*:Ñ*:Ó;HÜ Ø×ÑØ×Ñ Ñ(ØØ LLØ-ô ð r' c ó | j j dd« d }t |« }|rt j d|| « y t
|
j j |« \ }}}}}}|dk( rt j j t
|
j j |« « rL|j d« s|dz }t
|
j j |d« }t j d|« t! ||¬ « }t# || j$ ¬
|
« S # t& $ r t j d | « Y y t( $ r6}t j d | |j* |j, « Y d }~y d }~wt. $ r}t1 | |« Y d }~y d }~wt2 $ r}t1 | |« Y d }~y d }~wt4 $ r6}d } | t7 |« z } t1 | | t j8 ¬« Y d }~y d }~wt: j< $ r}t1 | d| « Y d }~y d }~wt: j> $ r t1 | d« Y y w xY w)Nú#r r zICannot look at %s URL %s because it does not support lookup as web pages.r ú/z
|
index.htmlz# file: URL is directory, getting %srX )rj z`Skipping page %s because it looks like an archive, and cannot be checked by a HTTP HEAD request.zºSkipping page %s because the %s request got Content-Type: %s. The only supported Content-Types are application/vnd.pypi.simple.v1+json, application/vnd.pypi.simple.v1+html, and text/htmlz4There was a problem confirming the ssl certificate: )r» zconnection error: z timed out) r# Úsplitr& r[ ÚwarningrL rM ÚurlparseÚosrR Úisdirr@ Ú url2pathnameÚendswithÚurljoinr\ r^ r¾ rj rF r) r/ r. r r¼ r r r2 Úinfor ÚConnectionErrorÚTimeout)
|
r rP r# Ú
|
vcs_schemer$ Ú_rR rU Úexcrº s
|
r% Ú_get_index_contentrÐ ? s Ø
|
((..aÓ
|
Ñ
|
#Cô # 3Ó'JÙÜØ WØ Ø ô
|
ð
|
ô &||× 4Ñ 4°SÓ 9Ñ FAtQ1Ø ÒBGGMM¬&¯.©.×*EÑ*EÀdÓ*KÔ Lð ||CÔ Ø 3JCô
|
ll×"Ñ" 3¨ Ó5Ü Ð:¸CÔ@ð UÜ# C°Ô9ô: # 4¸D×<SÑ<SÔTÐTøô9 ò
|
Üð 1à õ
|
ð8 ô/ ò
|
Üð Að Ø × Ñ Ø × Ñ ÷
|
ð
|
ð, ûô "ò +Ü c×*Ð*ð ûô ò +Ü c×*Ð*ð ûô ò @ØGØ#c(ÑÜ f´6·;±;×?Ñ?ð ûô × #Ñ #ò BÜ Ð(:¸3¸%Ð&@×AÐAð
|
ûô × Ñ ò 3Ü kÕ2ð ð 3úsT Ã9 D Ä H=Ä>H=Å,E7Å7 H=Æ FÆ H=Æ F1Æ1 H=Æ=,G.Ç.H=ÈHÈ"H=È<H=c ó" e Zd ZU ded< ded< y)ÚCollectedSourceszSequence[LinkSource | None]Ú
|
find_linksÚ
|
index_urlsN)r5 r6 r7 r§ rG r' r% rÒ rÒ | s
Ø+Ó+Ø+Ô+r' rÒ c óv e Zd ZdZ ddZe d d
|
d« Zed d« Zd dZ d dZ
|
y)Ú LinkCollectorzµ
|
Responsible for collecting Link objects from all configured locations,
|
making network requests as needed.
|
|
The class's main method is its collect_sources() method.
|
c ó || _ || _ y r+ )Ú search_scoperP )r0 rP rØ s r% r- zLinkCollector.__init__ s ð
|
)ÔØ r' c ó0 |j g|j z }|j r0|s.t j ddj d |D « « « g }|j xs g }t j |||j ¬« }t ||¬« }|S )zÆ
|
:param session: The Session to use to make requests.
|
:param suppress_no_index: Whether to ignore the --no-index option
|
when constructing the SearchScope object.
|
zIgnoring indexes: %sú,c 3 ó2 K | ] }t |« y wr+ r )Ú.0r# s r% ú <genexpr>z'LinkCollector.create.<locals>.<genexpr>¡ s è ø ÒI°sÔ-¨c×2ÑIùs )rÓ rÔ Úno_index)rP rØ )
|
Ú index_urlÚextra_index_urlsrÞ r[ r\ r] rÓ r ÚcreaterÖ )ÚclsrP ÚoptionsÚsuppress_no_indexrÔ rÓ rØ Úlink_collectors r% rá zLinkCollector.create s ð ×'Ñ'Ð(¨7×+CÑ+CÑC
|
Ø × Ò Ñ$5Ü LLØ&ØÑI¸jÔIÓIô ð Jð ×'Ñ'Ò-¨2
|
ä"×)Ñ)Ø!Ø!Ø ×%Ñ%ô
|
ô
|
'ØØ%ô
|
ð Ð r' c ó. | j j S r+ )rØ rÓ rt s r% rÓ zLinkCollector.find_links³ s à× Ñ ×+Ñ+Ð+r' c ó0 t || j ¬« S )z>
|
Fetch an HTML page containing package links.
|
rX )rÐ rP )r0 Úlocations r% Úfetch_responsez LinkCollector.fetch_response· s ô " (°D·L±LÔAÐAr' c ó t j fd j j « D « « j « }t j fd j
|
D « « j « }t j t j « rwt j ||« D cg c] }||j d|j ! }}t |« d dg|z }t j dj |« « t! t# |« t# |« ¬« S c c}w )Nc 3 óh K | ]) }t |j j d d ¬« + yw)F©Úcandidates_from_pageÚpage_validatorÚ
|
expand_dirrj Ú project_nameN©r rP Úis_secure_origin©rÜ Úlocrí rð r0 s r% rÝ z0LinkCollector.collect_sources.<locals>.<genexpr>Ã sA øè ø ò
|
4
|
ð ô ØØ%9Ø #||× <Ñ <Ø Ø#(Ø )÷ ð ñ
|
4
|
ùó /2c 3 óh K | ]) }t |j j d d ¬« + yw)Trì Nrñ ró s r% rÝ z0LinkCollector.collect_sources.<locals>.<genexpr>Î sA øè ø ò
|
5
|
ð ô ØØ%9Ø #||× <Ñ <Ø Ø#'Ø )÷ ð ñ
|
5
|
ùrõ z* z' location(s) to search for versions of ú:ú
|
)rÓ rÔ )Ú collectionsÚ OrderedDictrØ Úget_index_urls_locationsÚvaluesrÓ r[ Ú isEnabledForÚloggingÚDEBUGÚ itertoolsÚchainr r" r\ r] rÒ r )r0 rð rí Úindex_url_sourcesÚfind_links_sourcesÚsÚliness ``` r% Úcollect_sourcesz LinkCollector.collect_sources½ s2 ú ô (× 3Ñ 3õ
|
4
|
ð ×(Ñ(×AÑAÀ,ÓOô
|
4
|
ó
|
|
÷ &(ð ô )× 4Ñ 4õ
|
5
|
ð ô
|
5
|
ó
|
|
÷ &(ð ô × Ñ w}}Ô -ô # Ð);Ð=NÓOöàØ= Q§V¡VÐ%7ð QVVH ðEð ô u:,ð #Ø#/ .°ð3ðð ñEô LL 5Ó)Ô *ä ÜÐ .Ó/ÜÐ -Ó.ô
|
ð
|
ùòs Â>$D<N)rP r rØ r r3 r4 )F)rP r rã r rä rx r3 rÖ )r3 z list[str])rè r r3 úIndexContent | None)rð r2 rí r r3 rÒ ) r5 r6 r7 r¦ r- Ú classmethodrá ÚpropertyrÓ ré r rG r' r% rÖ rÖ s£ ñð àð ð "ð ð
|
ó ð ð
|
#(ð àð ð ð ð ð ð
|
|
ò ó ð ðB ò,ó ð,óBð ,
|
àð,
|
ð 1ð,
|
ð
|
ô ,
|
r' rÖ )r# r2 r3 r£ )rB r r3 r4 )r# r2 rP r r3 r4 )r# r2 rP r r3 r )r> ÚResponseHeadersr3 r£ )r r{ r3 r{ r~ r+ )r r rº zstr | Exceptionr» zCallable[..., None] | Noner3 r4 )T)rB r rj rx r3 rv )r r rP r r3 r )Or¦ Ú
|
__future__r rù Ú email.messagera r r r rþ rÅ Ú urllib.parserL Úurllib.requestÚcollections.abcr r r Ú dataclassesr Ú html.parserr Úoptparser Útypingr
|
r r Ú pip._vendorr Úpip._vendor.requestsr Ú pip._vendor.requests.exceptionsr r Úpip._internal.exceptionsr Úpip._internal.models.linkr Ú!pip._internal.models.search_scoper Ú pip._internal.network.sessionr Úpip._internal.network.utilsr Ú pip._internal.utils.filetypesr Úpip._internal.utils.miscr Úpip._internal.vcsr Úsourcesr r r Ú getLoggerr5 r[ r2 r
|
r& Ú Exceptionr) rD rF rV r^ rf rh r{ r r rv r r¼ r¾ rÐ rÒ rÖ rG r' r% ú<module>r" s ðñõ #ã Û Û Û Û Û Û Û Û ß >Ñ >Ý !Ý "Ý ÷ñ õ !Ý )ß @å ;Ý *Ý 9Ý 4Ý 8Ý 9Ý 9Ý !ç AÑ Aà × Ñ 8Ó $à c Ñ*óô)Yô )ó@ô, yô ó ó$<ó~÷ #ñ #ôBô Bóð( òó ðñ8 $Ô÷.ð .ó ð.ô(Zô ð> (,ð@Ø
|
ð@à ð@ð %ð@ð
|
ó @ð 48ð
|
Øð
|
Ø,0ð
|
àó
|
ó:ôz,zô ,÷
|
h
|
ò h
|
r'
|
