Lc
@ s5 d g Z d d k Z d d k Z d d k Z d d k Z d d k Z d d k l Z l Z l Z d d k
l Z l Z d Z
d Z d Z e i Z d Z e f d Z d e f d
YZ d e f d YZ d
e f d YZ d e f d YZ d e f d YZ d e f d YZ d S( t PooliN( t Processt cpu_countt TimeoutError( t Finalizet debugi i i c C s
t | S( N( t map( t args( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt mapstar& s c
C s | i } | i } t | d o | i i | i i n | d j o | | n x y
| } Wn$ t t f j
o t d Pn X| d j o t d Pn | \ } } } }
} y t
| |
| f } Wn! t j
o }
t |
f } n X| | | | f q^ d S( Nt _writers) worker got EOFError or IOError -- exitings worker got sentinel -- exiting(
t putt gett hasattrR t closet _readert Nonet EOFErrort IOErrorR t Truet Exceptiont False( t inqueuet outqueuet initializert initargsR
R t taskt jobt it funcR t kwdst resultt e( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt worker- s,
c B s e Z d Z e Z d d d d Z d Z d h d Z d d Z d d Z d d Z
d h d d Z d d d Z e
d
Z e
d Z e
d Z d
Z d Z d Z d Z e
d Z e d Z RS( sH
Class which supports an async version of the `apply()` builtin
c C s | i t i | _ h | _ t | _ | d j o- y
t } Wqe t j
o
d } qe Xn g | _ xy t
| D]k } | i d t d | i
| i | | f } | i i | | i i d d | _ t | _ | i q{ Wt i d t i d | i | i | i | i f | _ t | i _ t | i _ | i i t i d t i d | i | i | i f | _ t | i _ t | i _ | i i t | | i d | i | i
| i | i | i | i | i f d d | _ d S( Ni t targetR R t
PoolWorkert exitpriorityi (! t
_setup_queuest Queuet
_taskqueuet _cachet RUNt _stateR R t NotImplementedErrort _poolt rangeR R t _inqueuet _outqueuet appendt namet replaceR t daemont startt threadingt ThreadR t
_handle_taskst
_quick_putt
_task_handlert _handle_resultst
_quick_gett _result_handlerR t _terminate_poolt
_terminate( t selft processesR R R t w( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt __init__S sH
$
c C sP d d k l } | | _ | | _ | i i i | _ | i i i | _ d S( Ni ( t SimpleQueue(
t queuesRB R- R. R t sendR7 R t recvR: ( R> RB ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR$ s
c C s0 | i t j p t | i | | | i S( s1
Equivalent of `apply()` builtin
( R) R( t AssertionErrort apply_asyncR ( R> R R R ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt apply s c C s0 | i t j p t | i | | | i S( s/
Equivalent of `map()` builtin
( R) R( RF t map_asyncR ( R> R t iterablet chunksize( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR s i c s | i t j p t | d j oF t | i | i i f d t | D i f S| d j p t t i
| | } t | i | i i f d t | D i f d D Sd S( sZ
Equivalent of `itertools.imap()` -- can be MUCH slower than `Pool.map()`
i c 3 s4 x- | ]& \ } } i | | f h f Vq Wd S( N( t _job( t .0R t x( R R ( s* /usr/lib/python2.6/multiprocessing/pool.pys <genexpr> s c 3 s4 x- | ]& \ } } i | t | f h f Vq Wd S( N( RL R ( RM R RN ( R ( s* /usr/lib/python2.6/multiprocessing/pool.pys <genexpr> s c s s* x# | ] } x | D] } | Vq Wq Wd S( N( ( RM t chunkt item( ( s* /usr/lib/python2.6/multiprocessing/pool.pys <genexpr> s N( R) R( RF t IMapIteratorR' R& R
t enumeratet _set_lengthR t
_get_tasks( R> R RJ RK t task_batches( ( R R s* /usr/lib/python2.6/multiprocessing/pool.pyt imap s
c s | i t j p t | d j oF t | i | i i f d t | D i f S| d j p t t i
| | } t | i | i i f d t | D i f d D Sd S( sK
Like `imap()` method but ordering of results is arbitrary
i c 3 s4 x- | ]& \ } } i | | f h f Vq Wd S( N( RL ( RM R RN ( R R ( s* /usr/lib/python2.6/multiprocessing/pool.pys <genexpr> s c 3 s4 x- | ]& \ } } i | t | f h f Vq Wd S( N( RL R ( RM R RN ( R ( s* /usr/lib/python2.6/multiprocessing/pool.pys <genexpr> s c s s* x# | ] } x | D] } | Vq Wq Wd S( N( ( RM RO RP ( ( s* /usr/lib/python2.6/multiprocessing/pool.pys <genexpr> s N( R) R( RF t IMapUnorderedIteratorR' R& R
RR RS R RT ( R> R RJ RK RU ( ( R R s* /usr/lib/python2.6/multiprocessing/pool.pyt imap_unordered s
c C sX | i t j p t t | i | } | i i | i d | | | f g d f | S( s>
Asynchronous equivalent of `apply()` builtin
N( R) R( RF t ApplyResultR' R& R
RL R ( R> R R R t callbackR ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRG s +c s | i t j p t t | d p t | } n | d j oA t t | t | i d \ } } | o | d 7} q n t i
| | | } t | i | t | | | i
i f d t | D d f S( s<
Asynchronous equivalent of `map()` builtin
t __len__i i c 3 s4 x- | ]& \ } } i | t | f h f Vq Wd S( N( RL R ( RM R RN ( R ( s* /usr/lib/python2.6/multiprocessing/pool.pys <genexpr> s N( R) R( RF R t listR t divmodt lenR+ R RT t MapResultR' R& R
RR ( R> R RJ RK RZ t extraRU ( ( R s* /usr/lib/python2.6/multiprocessing/pool.pyRI s
( c
C s9 t i } x t | i d D] \ } } d } x t | D]T \ } } | i o t d Pn y | | Wq> t j
o t d Pq> Xq> W| o t d | | d q q Pq Wt d y@ t d | i d t d x | D] } | d q WWn t j
o t d n Xt d
d S( Nis' task handler found thread._state != RUNs could not put task on queues doing set_length()i s task handler got sentinels/ task handler sending sentinel to result handlers( task handler sending sentinel to workerss/ task handler got IOError when sending sentinelss task handler exiting(
R4 t current_threadt iterR R RR R) R R R
(
t taskqueueR
R t poolt threadt taskseqt
set_lengthR R t p( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR6 s<
c C s t i } x y
| } Wn$ t t f j
o t d d SX| i o& | i t j p t t d Pn | d j o t d Pn | \ } } } y | | i | | Wq t
j
o q Xq x | o | i t j o y
| } Wn$ t t f j
o t d d SX| d j o t d q n | \ } } } y | | i | | Wq t
j
o q Xq Wt | d oa t d y7 x0 t d D]" } | i
i p Pn | qWWqt t f j
o qXn t d t | | i d S( Ns. result handler got EOFError/IOError -- exitings, result handler found thread._state=TERMINATEs result handler got sentinels& result handler ignoring extra sentinelR s" ensuring that outqueue is not fulli
s7 result handler exiting: len(cache)=%s, thread._state=%s( R4 Ra R R R R) t TERMINATERF R t _sett KeyErrorR R, R t pollR^ ( R R t cacheRe R R R t obj( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR9 s\
c c sE t | } x2 t t i | | } | p d S| | f Vq d S( N( Rb t tuplet itertoolst islice( R t itt sizeRN ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRT 7 s c C s t d d S( Ns: pool objects cannot be passed between processes or pickled( R* ( R> ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt
__reduce__@ s c C s; t d | i t j o t | _ | i i d n d S( Ns closing pool( R R) R( t CLOSER& R
R ( R> ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR
E s
c C s! t d t | _ | i d S( Ns terminating pool( R Ri R) R= ( R> ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt terminateK s
c C sc t d | i t t f j p t | i i | i i x | i D] } | i qK Wd S( Ns joining pool( R R) Ru Ri RF R8 t joinR; R+ ( R> Rh ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRw P s
c C sZ t d | i i x< | i o. | i i o | i i t i d q Wd S( Ns7 removing tasks from inqueue until task handler finishedi ( R t _rlockt acquiret is_aliveR Rl RE t timet sleep( R t task_handlerRs ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt _help_stuff_finishX s
c C s: t d t | _ | i d t d | i | | t | | i p t | d j p t t | _ | i d | o= t | d d o) t d x | D] } | i
q Wn t d | i d t d | i d | o= t | d d o) t d x | D] } | i q Wn d S(
Ns finalizing pools& helping task handler/workers to finishi Rv s terminating workerss joining task handlerg}Ô%ITs joining result handlers joining pool workers( R Ri R) R
R R~ R^ Rz RF R Rv Rw ( t clsRc R R Rd R} t result_handlerRm Rh ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR< a s,
'
N( ( ( ( t __name__t
__module__t __doc__R R RA R$ RH R RV RX RG RI t staticmethodR6 R9 RT Rt R
Rv Rw R~ t classmethodR< ( ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR M s& . ': RY c B sD e Z d Z d Z d Z d d Z d d Z d Z RS( c C sS t i t i | _ t i | _ | | _ t | _ | | _
| | | i <d S( N( R4 t Conditiont Lockt _condt job_countert nextRL R' R t _readyt _callback( R> Rm RZ ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRA s c C s | i S( N( R ( R> ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt ready s c C s | i p t | i S( N( R RF t _success( R> ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt
successful s c C sD | i i z" | i p | i i | n Wd | i i Xd S( N( R Ry R t waitt release( R> t timeout( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR s
c C s@ | i | | i p
t n | i o | i S| i d S( N( R R R R t _value( R> R ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR s
c C s | \ | _ | _ | i o | i o | i | i n | i i z t | _ | i i Wd | i i X| i | i
=d S( N( R R R R Ry R R t notifyR R' RL ( R> R Rn ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRj s
N( R R RA R R R R R Rj ( ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRY s R_ c B s e Z d Z d Z RS( c C sw t i | | | t | _ d g | | _ | | _ | d j o d | _ t | _ n | | t | | | _ d S( Ni (
RY RA R R R R t
_chunksizet _number_leftR t bool( R> Rm RK t lengthRZ ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRA s
c C s | \ } } | o | | i | | i | d | i +| i d 8_ | i d j og | i o | i | i n | i | i =| i i z t | _ | i i
Wd | i i XqnX t | _
| | _ | i | i =| i i z t | _ | i i
Wd | i i Xd S( Ni i ( R R R R R' RL R Ry R R R R R R ( R> R t success_resultt successR ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRj s* "
( R R RA Rj ( ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR_ s RQ c B s> e Z d Z d Z d d Z e Z d Z d Z RS( c C sk t i t i | _ t i | _ | | _ t i | _
d | _ d | _
h | _ | | | i <d S( Ni ( R4 R R R R R RL R' t collectionst dequet _itemst _indexR t _lengtht _unsorted( R> Rm ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRA s c C s | S( N( ( R> ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt __iter__ s c C s | i i z y | i i } Wn t j
o | i | i j o
t n | i i | y | i i } Wq t j
o* | i | i j o
t n t q Xn XWd | i i
X| \ } } | o | S| d S( N( R Ry R t popleftt
IndexErrorR R t
StopIterationR R R ( R> R RP R t value( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR s&
c C s | i i z | i | j o | i i | | i d 7_ xL | i | i j o8 | i i | i } | i i | | i d 7_ qB W| i i n | | i | <| i | i j o | i | i
=n Wd | i i Xd S( Ni ( R Ry R R R/ R t popR R R' RL R ( R> R Rn ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRj s
c C s` | i i z> | | _ | i | i j o | i i | i | i =n Wd | i i Xd S( N( R Ry R R R R' RL R ( R> R ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRS s
N( R R RA R R R t __next__Rj RS ( ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRQ s
RW c B s e Z d Z RS( c C sv | i i zT | i i | | i d 7_ | i i | i | i j o | i | i =n Wd | i i Xd S( Ni (
R Ry R R/ R R R R' RL R ( R> R Rn ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRj / s
( R R Rj ( ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRW - s t
ThreadPoolc B sB e Z d d k l Z d d d d Z d Z e d Z RS( i ( R c C s t i | | | | d S( N( R RA ( R> R? R R ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRA B s c C s@ t i | _ t i | _ | i i | _ | i i | _ d S( N( R% R- R. R
R7 R R: ( R> ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR$ E s c C sW | i i z5 | i i | i i d g | | i i Wd | i i Xd S( N( t not_emptyRy t queuet cleart extendR t
notify_allR ( R R} Rs ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR~ K s
N( ( R R t dummyR R RA R$ R R~ ( ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR > s ( t __all__R4 R% Rp R R{ t multiprocessingR R R t multiprocessing.utilR R R( Ru Ri t countR R R R t objectR RY R_ RQ RW R ( ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt <module> s( :2,I
y~or5J ={Eeu磝Qk ᯘG{?+]ן?wM3X^歌>{7پK>on\jy R g/=fOroNVv~Y+ NGuÝHWyw[eQʨSb> >}Gmx[o[<{Ϯ_qF vM IENDB`
