$9ii% S SK Jr S SKJr S SKJrJr S SKJr S SKr S SK
J r S SK J
r
S SKJr S S KJr S S
KJr S S KJr \ " S S
5 5 r " S S5 rg) ) annotations) dataclass)datetimedate)TupleN) webdriver)By)Service)
WebDriverWait)expected_conditions)ChromeDriverManagerc * \ rS rSr% S\S' S\S' Srg) ScrapeResult pd.DataFramesalidasllegadas N)__name__
__module__ __qualname____firstlineno____annotations____static_attributes__r aC:\Users\julio\OneDrive\Documentos\Trabajo\Ideas Frescas\Proyectos\Vuelos_Mazatlan\src\scraper.pyr r s
r r c \ rS rSrSrSSS jjrSS jrSSS jjrSS jrSS jr SSS jjr
\ SS j5 r SS
jr
SS jrS rg
)OMAFlightScraper u
Scraper dinámico para OMA Mazatlán (estatus de vuelos).
HTML real: filas son divs .table-row.departure / .table-row.arrival,
celdas son .item__1 ... .item__6
c Xl X0l [ R " 5 nU( a UR S5 UR S5 UR S5 UR S5 UR S5 [ [
5 R 5 5 n[ R " XTS9U l [ U R U R 5 U l g )Nz--headless=newz --no-sandboxz--disable-dev-shm-usagez--window-size=1400,900z --lang=es-MX)serviceoptions) urltimeoutr
ChromeOptions add_argument
ChromeServicer
installChromedriverr wait)selfr# headlesstimeout_secondsr" r! s r __init__OMAFlightScraper.__init__ s & ))+ !1 2 ^, 67 56 ^, 3 5 = = ?@ &&wH !$++t||< r c Z U R R 5 g ! [ a g f = fN)r* quit Exception)r, s r closeOMAFlightScraper.close. s) KK s
**c [ U5 Hk n U R R [ R " U5 5 nU R
R
SU5 U R
R
SU5 g g! [ a M} f = f)N.arguments[0].scrollIntoView({block:'center'});arguments[0].click();TF)ranger+ untilECelement_to_be_clickabler* execute_scriptr4 )r, locatortries_els r _safe_click OMAFlightScraper._safe_click7 sy u A
YY__R%?%?%HI **+[]_` **+BBG
s A'A<<
B
B
c & [ R S4[ R S4[ R S4/nU HE n U R R " U6 nU( a! U R R SUS 5 g MG g ! [ a MW f = f)Nz //button[contains(.,'Aceptar')]z //button[contains(.,'ACEPTAR')]z"button#onetrust-accept-btn-handlerr9 r )r XPATH CSS_SELECTORr*
find_elementsr> r4 )r,
candidateslocbtnss r _dismiss_cookie_banner_if_any.OMAFlightScraper._dismiss_cookie_banner_if_anyB s
XX8 9
XX8 9
__ B C
C
{{00#6KK../FQP
s ?B
BBc UR 5 R S5 ( a [ R S4nO[ R S4nU R USS9 UR 5 R S5 ( a@ U R
R
[ R " [ R S45 5 gU R
R
[ R " [ R S45 5 g) zH
tab: "SALIDAS" o "LLEGADAS"
En HTML: #departureChange y #arrivalChange
SALz#departureChangez#arrivalChange )r@ z.table-row.departurez.table-row.arrivalN) upper
startswithr rG rC r+ r; r< presence_of_element_located)r, tabrJ s r
_go_to_tabOMAFlightScraper._go_to_tabT s
99; ! !% ( (??$67C??$45C A& 99; ! !% ( ( IIOOB : :BOOMc;d e f IIOOB : :BOOMa;b c dr c ^ ^^ S U 4S jjmT" 5 m[ U5 H nT R R [ R S5 nU( d g T R R SUS 5 T R R SUS 5 [ T R S5 R UU4S j5 T" 5 mM g! [ a gf = f! [ a gf = f)
u
En tu HTML: VER MÁS VUELOS
A veces carga más filas vía AJAX o cambia el DOM.
Estrategia:
- medir conteo de filas totales (arrival+departure)
- click al link mientras crezca el conteo
c > TR R [ R S5 n U Vs/ s HE nSUR S5 =( d S; d M# SUR S5 =( d S; d MC UPMG n n[ U 5 $ s snf )NzF.table-wrapper .table-row.departure, .table-wrapper .table-row.arrival table-headerclass table-title-row)r* rH r rG
get_attributelen)rowsrr, s r count_all_rows.count_all_rowsr s ;;,,R__ ?G HD # ]t!~aoog>V>\Z\']Abs|} }L }L MT }U }[ Y[ c\AtD ]t9 ]s "B B 6B z a#allFlights.table-bottom-linkr8 r r9 c > T" 5 T: $ r2 r )dra prevs r 6OMAFlightScraper._expand_all_flights.. s n>NQU>Ur N)returnint) r: r* rH r rG r> r4 r r; )r,
max_clicksrA linkra rf s ` @@r _expand_all_flights$OMAFlightScraper._expand_all_flightsj s z"A;;,,R__>^_D
**+[]abc]de **+BDGL
dkk1-334UV%' #
s$ >C
1C
CC
C)(C)c U R [ R S5 nUR =( d SR 5 nU( a U$ UR [ R S5 nUR
S5 =( d SR 5 nU( a U$ UR
S5 =( d SR 5 nU( a UR S5 S R S5 S nUR S
S5 R S S 5 R S
S 5 R 5 n[ U5 S:: a UR 5 $ UR 5 $ g! [ a gf = f! [ a gf = f)u
La aerolínea puede venir como:
- texto directo
- img src=/img/logo_xxx.png (sin alt)
En ese caso tomamos el filename del src como fallback.
z.item__2r[ imgaltsrc/.r logo_- rA ) find_elementr rG r4 textstripr] splitreplacer^ rQ title)row_elcelltxtrp rq rr names r _get_airline_name"OMAFlightScraper._get_airline_name sG &&r
CD yyB%%' J ##BOOU;C$$U+1r88:C
$$U+1r88:Cyy~b)//4Q7||GR088cBJJ3PSTZZ\'*4yA~tzz|G4::<G - & s1 E& AE6 B3E6 E6 &
E32E36
FFc ^ US:X a SOSnU R R [ R " [ R
U45 5 U R R [ R
U5 n/ nU HD mTR S5 =( d SnSU; a M% SU; d SU; a M3 UR T5 MF / nU H mSU4S jjnU" S
5 nU R T5 n U" S 5 n
U" S 5 n U" S
5 n U" S5 n
U( d U
( d U ( d M[ UU U
U U
S.nUS:X a XS' OXS' UR U5 M [ R " U5 n[ R " 5 R 5 n[ R" " 5 R% S5 nUR& ( d& UR) SSU5 UR) SSU5 U$ )u{
mode: "salidas" o "llegadas"
Usa:
- .table-row.departure
- .table-row.arrival
Excluye nullfs (No hay información...)
r z#.table-wrapper .table-row.departurez!.table-wrapper .table-row.arrivalrZ r[ nullfsrY r\ c > TR [ R U 5 R =( d SR 5 $ ! [
a gf = f)Nr[ )rz r rG r{ r| r4 )cssr` s r cell_text1OMAFlightScraper._extract_rows..cell_text sC NN2??C @ E E KRRTT s A A
AAz.item__1z.item__3z.item__4z.item__5z.item__6)Hora AerolineaNoVueloTerminalEstatusDestinoOrigenz%Y-%m-%d %H:%M:%Sr Fecha ScrapedAt)r strri r )r+ r; r< rS r rG r* rH r] appendr pd DataFramer today isoformatr nowstrftimeemptyinsert)r, modeselr_ cleanedclsdatar hora aerolineano_vuelo dest_origenterminalestatusrowdfr
scraped_atr` s @r
_extract_rows OMAFlightScraper._extract_rows s 8 A??7+1rC3 $(9S(@ NN1
A
Z(D ..q1I ,H#J/K ,H
+G H &# $" C y !,I +H
KK ? B \\$
&&( \\^,,-@A
xx IIa% ( IIaj 1 r c U R R U R 5 U R 5 U R R [ R " [ R S45 5 U R 5 U R S5 U R S5 nU R S5 U R S5 n[ XS9$ )Nz.table-wrapperSALIDASr LLEGADASr )r r )r* getr# rL r+ r; r< rS r rG rm rU r r )r, r r s r
scrape_allOMAFlightScraper.scrape_all s ! **,
66IY7Z[\ "
"$$Y/
#%%j1G??r )r* r$ r# r+ N)T )r# r r- boolr. rj ri None)ri r ) )r? zTuple[By, str]r@ rj ri r )rT r ri r ) )rk rj ri r )ri r )r r ri r )ri r )r r r r __doc__r/ r5 rC rL rU rm staticmethodr r r r r r r r r sI =
$e,!L BCP@r r )
__future__r dataclassesr r r typingr pandasr seleniumr selenium.webdriver.common.byr !selenium.webdriver.chrome.servicer
r' selenium.webdriver.support.uir selenium.webdriver.supportr r< webdriver_manager.chromer
r r r r r r sM " ! # + F 7 @ 8
s@ s@r