Lc
@ s5 d g Z d d k Z d d k Z d d k Z d d k Z d d k Z d d k l Z l Z l Z d d k
l Z l Z d Z
d Z d Z e i Z d Z e f d Z d e f d
YZ d e f d YZ d
e f d YZ d e f d YZ d e f d YZ d e f d YZ d S( t PooliN( t Processt cpu_countt TimeoutError( t Finalizet debugi i i c C s
t | S( N( t map( t args( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt mapstar& s c
C s | i } | i } t | d o | i i | i i n | d j o | | n x y
| } Wn$ t t f j
o t d Pn X| d j o t d Pn | \ } } } }
} y t
| |
| f } Wn! t j
o }
t |
f } n X| | | | f q^ d S( Nt _writers) worker got EOFError or IOError -- exitings worker got sentinel -- exiting(
t putt gett hasattrR t closet _readert Nonet EOFErrort IOErrorR t Truet Exceptiont False( t inqueuet outqueuet initializert initargsR
R t taskt jobt it funcR t kwdst resultt e( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt worker- s,
c B s e Z d Z e Z d d d d Z d Z d h d Z d d Z d d Z d d Z
d h d d Z d d d Z e
d
Z e
d Z e
d Z d
Z d Z d Z d Z e
d Z e d Z RS( sH
Class which supports an async version of the `apply()` builtin
c C s | i t i | _ h | _ t | _ | d j o- y
t } Wqe t j
o
d } qe Xn g | _ xy t
| D]k } | i d t d | i
| i | | f } | i i | | i i d d | _ t | _ | i q{ Wt i d t i d | i | i | i | i f | _ t | i _ t | i _ | i i t i d t i d | i | i | i f | _ t | i _ t | i _ | i i t | | i d | i | i
| i | i | i | i | i f d d | _ d S( Ni t targetR R t
PoolWorkert exitpriorityi (! t
_setup_queuest Queuet
_taskqueuet _cachet RUNt _stateR R t NotImplementedErrort _poolt rangeR R t _inqueuet _outqueuet appendt namet replaceR t daemont startt threadingt ThreadR t
_handle_taskst
_quick_putt
_task_handlert _handle_resultst
_quick_gett _result_handlerR t _terminate_poolt
_terminate( t selft processesR R R t w( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt __init__S sH
$
c C sP d d k l } | | _ | | _ | i i i | _ | i i i | _ d S( Ni ( t SimpleQueue(
t queuesRB R- R. R t sendR7 R t recvR: ( R> RB ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR$ s
c C s | i | | | i S( s1
Equivalent of `apply()` builtin
( t apply_asyncR ( R> R R R ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt apply s c C s | i | | | i S( s/
Equivalent of `map()` builtin
( t map_asyncR ( R> R t iterablet chunksize( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR s i c s | d j oF t | i | i i f d t | D i f St i | | } t | i | i i f d t | D i f d D Sd S( sZ
Equivalent of `itertools.imap()` -- can be MUCH slower than `Pool.map()`
i c 3 s4 x- | ]& \ } } i | | f h f Vq Wd S( N( t _job( t .0R t x( R R ( s* /usr/lib/python2.6/multiprocessing/pool.pys <genexpr> s c 3 s4 x- | ]& \ } } i | t | f h f Vq Wd S( N( RK R ( RL R RM ( R ( s* /usr/lib/python2.6/multiprocessing/pool.pys <genexpr> s c s s* x# | ] } x | D] } | Vq Wq Wd S( N( ( RL t chunkt item( ( s* /usr/lib/python2.6/multiprocessing/pool.pys <genexpr> s N( t IMapIteratorR' R& R
t enumeratet _set_lengthR t
_get_tasks( R> R RI RJ t task_batches( ( R R s* /usr/lib/python2.6/multiprocessing/pool.pyt imap s
c s | d j oF t | i | i i f d t | D i f St i | | } t | i | i i f d t | D i f d D Sd S( sK
Like `imap()` method but ordering of results is arbitrary
i c 3 s4 x- | ]& \ } } i | | f h f Vq Wd S( N( RK ( RL R RM ( R R ( s* /usr/lib/python2.6/multiprocessing/pool.pys <genexpr> s c 3 s4 x- | ]& \ } } i | t | f h f Vq Wd S( N( RK R ( RL R RM ( R ( s* /usr/lib/python2.6/multiprocessing/pool.pys <genexpr> s c s s* x# | ] } x | D] } | Vq Wq Wd S( N( ( RL RN RO ( ( s* /usr/lib/python2.6/multiprocessing/pool.pys <genexpr> s N( t IMapUnorderedIteratorR' R& R
RQ RR R RS ( R> R RI RJ RT ( ( R R s* /usr/lib/python2.6/multiprocessing/pool.pyt imap_unordered s
c C sA t | i | } | i i | i d | | | f g d f | S( s>
Asynchronous equivalent of `apply()` builtin
N( t ApplyResultR' R& R
RK R ( R> R R R t callbackR ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRF s +c s t | d p t | } n | d j oA t t | t | i d \ } } | o | d 7} qn n t i | | | } t | i | t | | | i
i f d t | D d f S( s<
Asynchronous equivalent of `map()` builtin
t __len__i i c 3 s4 x- | ]& \ } } i | t | f h f Vq Wd S( N( RK R ( RL R RM ( R ( s* /usr/lib/python2.6/multiprocessing/pool.pys <genexpr> s N(
R t listR t divmodt lenR+ R RS t MapResultR' R& R
RQ ( R> R RI RJ RY t extraRT ( ( R s* /usr/lib/python2.6/multiprocessing/pool.pyRH s
( c
C s9 t i } x t | i d D] \ } } d } x t | D]T \ } } | i o t d Pn y | | Wq> t j
o t d Pq> Xq> W| o t d | | d q q Pq Wt d y@ t d | i d t d x | D] } | d q WWn t j
o t d n Xt d
d S( Nis' task handler found thread._state != RUNs could not put task on queues doing set_length()i s task handler got sentinels/ task handler sending sentinel to result handlers( task handler sending sentinel to workerss/ task handler got IOError when sending sentinelss task handler exiting(
R4 t current_threadt iterR R RQ R) R R R
(
t taskqueueR
R t poolt threadt taskseqt
set_lengthR R t p( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR6 s<
c C s t i } x y
| } Wn$ t t f j
o t d d SX| i o t d Pn | d j o t d Pn | \ } } } y | | i | | Wq t j
o q Xq x | o | i t j o y
| } Wn$ t t f j
o t d d SX| d j o t d q n | \ } } } y | | i | | Wq t j
o q Xq Wt
| d oa t d y7 x0 t d D]" } | i i
p Pn | qWWqt t f j
o qXn t d t | | i d S( Ns. result handler got EOFError/IOError -- exitings, result handler found thread._state=TERMINATEs result handler got sentinels& result handler ignoring extra sentinelR s" ensuring that outqueue is not fulli
s7 result handler exiting: len(cache)=%s, thread._state=%s( R4 R` R R R R) R t _sett KeyErrort TERMINATER R, R t pollR] ( R R t cacheRd R R R t obj( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR9 sZ
c c sE t | } x2 t t i | | } | p d S| | f Vq d S( N( Ra t tuplet itertoolst islice( R t itt sizeRM ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRS 7 s c C s t d d S( Ns: pool objects cannot be passed between processes or pickled( R* ( R> ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt
__reduce__@ s c C s; t d | i t j o t | _ | i i d n d S( Ns closing pool( R R) R( t CLOSER& R
R ( R> ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR
E s
c C s! t d t | _ | i d S( Ns terminating pool( R Rj R) R= ( R> ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt terminateK s
c C sF t d | i i | i i x | i D] } | i q. Wd S( Ns joining pool( R R8 t joinR; R+ ( R> Rg ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRv P s
c C sZ t d | i i x< | i o. | i i o | i i t i d q Wd S( Ns7 removing tasks from inqueue until task handler finishedi ( R t _rlockt acquiret is_aliveR Rk RE t timet sleep( R t task_handlerRr ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt _help_stuff_finishX s
c C s t d t | _ | i d t d | i | | t | t | _ | i d | o= t | d d o) t d x | D] } | i q Wn t d | i d t d | i d | o= t | d d o) t d x | D] } | i q Wn d S(
Ns finalizing pools& helping task handler/workers to finishi Ru s terminating workerss joining task handlerg}Ô%ITs joining result handlers joining pool workers(
R Rj R) R
R R} R] R Ru Rv ( t clsRb R R Rc R| t result_handlerRl Rg ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR< a s*
N( ( ( ( t __name__t
__module__t __doc__R R RA R$ RG R RU RW RF RH t staticmethodR6 R9 RS Rs R
Ru Rv R} t classmethodR< ( ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR M s& . ': RX c B sD e Z d Z d Z d Z d d Z d d Z d Z RS( c C sS t i t i | _ t i | _ | | _ t | _ | | _
| | | i <d S( N( R4 t Conditiont Lockt _condt job_countert nextRK R' R t _readyt _callback( R> Rl RY ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRA s c C s | i S( N( R ( R> ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt ready s c C s | i S( N( t _success( R> ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt
successful s c C sD | i i z" | i p | i i | n Wd | i i Xd S( N( R Rx R t waitt release( R> t timeout( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR s
c C s@ | i | | i p
t n | i o | i S| i d S( N( R R R R t _value( R> R ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR s
c C s | \ | _ | _ | i o | i o | i | i n | i i z t | _ | i i Wd | i i X| i | i
=d S( N( R R R R Rx R R t notifyR R' RK ( R> R Rm ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRh s
N( R R RA R R R R R Rh ( ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRX s R^ c B s e Z d Z d Z RS( c C sw t i | | | t | _ d g | | _ | | _ | d j o d | _ t | _ n | | t | | | _ d S( Ni (
RX RA R R R R t
_chunksizet _number_leftR t bool( R> Rl RJ t lengthRY ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRA s
c C s | \ } } | o | | i | | i | d | i +| i d 8_ | i d j og | i o | i | i n | i | i =| i i z t | _ | i i
Wd | i i XqnX t | _
| | _ | i | i =| i i z t | _ | i i
Wd | i i Xd S( Ni i ( R R R R R' RK R Rx R R R R R R ( R> R t success_resultt successR ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRh s* "
( R R RA Rh ( ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR^ s RP c B s> e Z d Z d Z d d Z e Z d Z d Z RS( c C sk t i t i | _ t i | _ | | _ t i | _
d | _ d | _
h | _ | | | i <d S( Ni ( R4 R R R R R RK R' t collectionst dequet _itemst _indexR t _lengtht _unsorted( R> Rl ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRA s c C s | S( N( ( R> ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt __iter__ s c C s | i i z y | i i } Wn t j
o | i | i j o
t n | i i | y | i i } Wq t j
o* | i | i j o
t n t q Xn XWd | i i
X| \ } } | o | S| d S( N( R Rx R t popleftt
IndexErrorR R t
StopIterationR R R ( R> R RO R t value( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR s&
c C s | i i z | i | j o | i i | | i d 7_ xL | i | i j o8 | i i | i } | i i | | i d 7_ qB W| i i n | | i | <| i | i j o | i | i
=n Wd | i i Xd S( Ni ( R Rx R R R/ R t popR R R' RK R ( R> R Rm ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRh s
c C s` | i i z> | | _ | i | i j o | i i | i | i =n Wd | i i Xd S( N( R Rx R R R R' RK R ( R> R ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRR s
N( R R RA R R R t __next__Rh RR ( ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRP s
RV c B s e Z d Z RS( c C sv | i i zT | i i | | i d 7_ | i i | i | i j o | i | i =n Wd | i i Xd S( Ni (
R Rx R R/ R R R R' RK R ( R> R Rm ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRh / s
( R R Rh ( ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRV - s t
ThreadPoolc B sB e Z d d k l Z d d d d Z d Z e d Z RS( i ( R c C s t i | | | | d S( N( R RA ( R> R? R R ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyRA B s c C s@ t i | _ t i | _ | i i | _ | i i | _ d S( N( R% R- R. R
R7 R R: ( R> ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR$ E s c C sW | i i z5 | i i | i i d g | | i i Wd | i i Xd S( N( t not_emptyRx t queuet cleart extendR t
notify_allR ( R R| Rr ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR} K s
N( ( R R t dummyR R RA R$ R R} ( ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyR > s ( t __all__R4 R% Ro R Rz t multiprocessingR R R t multiprocessing.utilR R R( Rt Rj t countR R R R t objectR RX R^ RP RV R ( ( ( s* /usr/lib/python2.6/multiprocessing/pool.pyt <module> s( :2,I
y~or5J ={Eeu磝Qk ᯘG{?+]ן?wM3X^歌>{7پK>on\jy R g/=fOroNVv~Y+ NGuÝHWyw[eQʨSb> >}Gmx[o[<{Ϯ_qF vM IENDB`
