Since the worker is sleeping for a random amount of time, the output from this program may vary. It should look something like this:. At start-up, a Thread does some basic initialization and then calls its run method, which calls the target function passed to the constructor. To create a subclass of Thread , override run to do whatever is necessary. The return value of run is ignored.
Because the args and kwargs values passed to the Thread constructor are saved in private variables, they are not easily accessed from a subclass. To pass arguments to a custom thread type, redefine the constructor to save the values in an instance attribute that can be seen in the subclass. MyThreadWithArgs uses the same API as Thread , but another class could easily change the constructor method to take more or different arguments more directly related to the purpose of the thread, as with any other class.
One example of a reason to subclass Thread is provided by Timer , also included in threading. A Timer starts its work after a delay, and can be canceled at any point within that delay time period. Notice that the second timer is never run, and the first timer appears to run after the rest of the main program is done. Since it is not a daemon thread, it is joined implicitly when the main thread is done.
Although the point of using multiple threads is to spin separate operations off to run concurrently, there are times when it is important to be able to synchronize the operations in two or more threads. A simple way to communicate between threads is using Event objects. An Event manages an internal flag that callers can either set or clear. Other threads can wait for the flag to be set , effectively blocking progress until allowed to continue.
The wait method takes an argument representing the number of seconds to wait for the event before timing out. It returns a boolean indicating whether or not the event is set, so the caller knows why wait returned. The isSet method can be used separately on the event without fear of blocking. In addition to synchronizing the operations of threads, it is also important to be able to control access to shared resources to prevent corruption or missed data.
To guard against simultaneous access to an object, use a Lock object. In this example, the worker function increments a Counter instance, which manages a Lock to prevent two threads from changing its internal state at the same time. If the Lock was not used, there is a possibility of missing a change to the value attribute. To find out whether another thread has acquired the lock without holding up the current thread, pass False for the blocking argument to acquire. In the next example, worker tries to acquire the lock three separate times, and counts how many attempts it has to make to do so.
It takes worker more than three iterations to acquire the lock three separate times. Normal Lock objects cannot be acquired more than once, even by the same thread. This can introduce undesirable side-effects if a lock is accessed by more than one function in the same call chain. In this case, since both functions are using the same global lock, and one calls the other, the second acquisition fails and would have blocked using the default arguments to acquire.
A new lock is created by calling the Lock method, which returns the new lock. The acquire blocking method of the new lock object is used to force threads to run synchronously. The optional blocking parameter enables you to control whether the thread waits to acquire the lock. If blocking is set to 0, the thread returns immediately with a 0 value if the lock cannot be acquired and with a 1 if the lock was acquired.
If blocking is set to 1, the thread blocks and wait for the lock to be released. The release method of the new lock object is used to release the lock when it is no longer required. The Queue module allows you to create a new queue object that can hold a specific number of items. Python - Multithreaded Programming Advertisements. Previous Page. I chose eight worker threads because my computer has eight CPU cores and one worker thread per core seemed a good number for how many threads to run at once.
In practice, this number is chosen much more carefully based on other factors, such as other applications and services running on the same machine. This is almost the same as the previous one, with the exception that we now have a new class, DownloadWorker , which is a descendent of the Python Thread class. The run method has been overridden, which runs an infinite loop. On every iteration, it calls self. It blocks until there is an item in the queue for the worker to process. After the download is finished, the worker signals the queue that that task is done.
This is very important, because the Queue keeps track of how many tasks were enqueued. The call to queue. Running this Python threading example script on the same machine used earlier results in a download time of 4. While this is much faster, it is worth mentioning that only one thread was executing at a time throughout this process due to the GIL. Therefore, this code is concurrent but not parallel.
The reason it is still faster is because this is an IO bound task. The processor is hardly breaking a sweat while downloading these images, and the majority of the time is spent waiting for the network. This is why Python multithreading can provide a large speed increase. The processor can switch between the threads whenever one of them is ready to do some work. Using the threading module in Python or any other interpreted language with a GIL can actually result in reduced performance.
If your code is performing a CPU bound task, such as decompressing gzip files, using the threading module will result in a slower execution time. For CPU bound tasks and truly parallel execution, we can use the multiprocessing module. For example, IronPython, a Python implementation using the. You can find a list of working Python implementations here.
The only changes we need to make are in the main function. To use multiple processes, we create a multiprocessing Pool. With the map method it provides, we will pass the list of URLs to the pool, which in turn will spawn eight new processes and use each one to download the images in parallel.
This is true parallelism, but it comes with a cost. The entire memory of the script is copied into each subprocess that is spawned. While the threading and multiprocessing modules are great for scripts that are running on your personal computer, what should you do if you want the work to be done on a different machine, or you need to scale up to more than the CPU on one machine can handle?
A great use case for this is long-running back-end tasks for web applications. This will degrade the performance of your application for all of your users. What would be great is to be able to run these jobs on another machine, or many other machines. A great Python library for this task is RQ , a very simple yet powerful library. You first enqueue a function and its arguments using the library. This pickles the function call representation, which is then appended to a Redis list.
Enqueueing the job is the first step, but will not do anything yet. We also need at least one worker to listen on that job queue. The first step is to install and run a Redis server on your computer, or have access to a running Redis server.
After that, there are only a few small changes made to the existing code. We first create an instance of an RQ Queue and pass it an instance of a Redis server from the redis-py library. The enqueue method takes a function as its first argument, then any other arguments or keyword arguments are passed along to that function when the job is actually executed. One last step we need to do is to start up some workers.
RQ provides a handy script to run workers on the default queue. Just run rqworker in a terminal window and it will start a worker listening on the default queue. Please make sure your current working directory is the same as where the scripts reside in.
The great thing about RQ is that as long as you can connect to Redis, you can run as many workers as you like on as many different machines as you like; therefore, it is very easy to scale up as your application grows. Here is the source for the RQ version:.
0コメント