Building A Scalable Queueing System With PHP
Published on Feb 14, 2011
In today's article we are going to cover building a queueing system with PHP. Before we begin, let's define what a queueing system is. The best place to start is the dictionary:
"to line up or wait in a queue"
Now that we have our definition, let's define why we would want to build a queueing system. A queueing system is an excellent tool that will allow us to take a specific process and perform the functionality "offline", e.g. the process will line up and we will process them one at a time at a later date. This will probably be easier to explain with an example.
Imagine an admin area of a website that allows the administrator to send out a mass email to all of their users. The simple process to building this functionality would be as follows:
1. Build a form that accepts a subject and a body for the email.
2. Retrieve the list of users from your database.
3. Loop through the users and send each person an individual email.
The above example works nice and fast when there are only a few hundred users. However, imagine trying to send this email to 10,000 users. The administrator would be waiting a long time for this process to finish. Not only that, if they closed the browser, it probably would not finish properly.
So, the goal of our queueing system is to remove a specific process from running "online" (in a web browser) and running it "offline" with a scheduled task.