| |
|
 |
Inter-Process Communication in PHP
|
Reading and writing data
To write data to a shared memory segment you use:
int shmop_write (int shmid, string data, int offset) |
The shmid is the id returned by shmop_open, data is data and offset is the offset inside the segment (0 to start writing from the beginning of the segment.
To read you use:
string shmop_read (int shmid, int start, int count) |
You indicate the position (offset) from which you are going to read (0=start of segment) and the number of bytes. using shared memory you just use the memory as an array of bytes, you read "x" bytes, write "y" bytes, etc. You may want to program a wraper to store strings, objects or so and let the class manage the lengths using some standard format to put and retrieve things from the segment
Two processes
You'll have absolutely no problem accesing, creating, reading and writing a shared segment from one isolated PHP process, but you may run into trouble using more than 1 at the same time, you are going to enter the very interesting world of concurrency and the mutual exclusion problem
Suposse you have two processes reading and writing data in a segment, if two processes try to write data at the same time you might get garbage in the segment, you need to prevent more that one process from writing to the segment at the same time, this is known as the "mutual exclusion" problem, there's a lot of literature mainly in operating system couses regarding this subject. One easy way to achieve mutual exclusion is using semaphores, so we are going to cover this here:
Semaphores are another IPC resource, you can check them and remove them using ipcs and ipcrm too. You create a semaphore using:
int sem_get (int key [, int max_acquire [, int perm]]) |
The semaphore is created if necessary, the function returns an identifier used to manipulate the semaphore, max_acquire indicate the maximum number of processes that can acquire the semaphore without releasing it (default=1), permissions are set as usual.
Once you have a semaphore you can do two things with it: "acquire it" and "release it", when you acquire a semaphore you can think of it as if you were incrementing it. If you try to increment the semaphore past "max_acquire" the process will block until the call can be completed. In binary semaphores where max_acquire is 1 only one process can have the semaphore "acquired" at the same time, other processes trying to acquire the resource will block until the process with the semaphore releases it
To acquire/release the semaphore you use:
int sem_acquire (int sem_identifier)
int sem_release (int sem_identifier) |
Where the identifier is the one you got with sem_get.
A simple mutual esclusion protocol
You can use the following protocol to achieve mutual exclusion in PHP
$semid=sem_get(0xee3,1,0666);
$shm_id = shmop_open(0xff3, "c", 0644, 100);
sem_acquire($semid);
/* If we are here we are alone! */
WRITE TO THE SHARED SEGMENT HERE
sem_release($semid); |
As you can see the mechanism is simple, acquire the semaphore, do something, release the semaphore, you can be sure that two processes can't be writing at the same time because once one of them acquire the semaphore the other will block until the semaphore is released. The lines between the sem_acquire and sem_release are called "the critical zone", the zone where you don't want two or more processes at the same time
One interesting observation is that in PHP semaphores implementation the process releasing the semaphore must be the same that acquired it, this is not usual in IPC where a process may acquire a semaphore and let other process release it.
You should note that you only have to take care of mutual exclusion between writers, more than one process reading the same segment don't imply any risk of inconsitency.
Applications
There're several applications for IPC, from the simple action to keep a parsed configuration file shared between processes to advanced engines and authentication systems. I've used this extensions to optimize a bottle-neck in a web site keeping a big file that all the scripts needed to read in memory, you can find some problems that might be solved using this or even create powerful applications with this in mind.
Comment List
| Topic: |
Author: |
Time: |
|
Here's a working example
|
Jez Humble
|
31.10.2002 19:08
|
|
<?
/**
* The following four functions implement a permanent
* hashtable. This hash has an application scope,
* i.e. its contents can be accessed from all php scripts
* running on this computer.
*
* Furthermore, it is cached in the db, i.e. it's
* inviolable. This is designed to keep configuration settings
* for the site. Its contents are buffered in memory, so there's
* no hit on the db when accessing the hash apart from the first
* time you read it, and of course when you write to it.
*
* php must be compiled with --enable-shmop and --enable-sysvsem
* Thanks to Luis Argerich for the idea:
* http://zez.org/article/articleview/46/1/
*
* NB you also need to set up the values to access the db
* in the function below, and create a table
* with a field called settings of type text.
* the name of the database and table should be inserted
* into the appropriate 'define's below (SHM_DB, SHM_DB_TABLE).
*/
function get_db_connection($db)
{
$link = mysql_connect("localhost", 'USERNAME', 'PASSWORD');
mysql_select_db($db, $link);
return $link;
}
define("SHM_IDENTIFIER", 0x8ea);
// Set size to max size of a mysql 'text' field (64k, minus a few to allow for slashes)
// It would be nice to be able to increase the size automatically to accommodate larger
// hashes, but I don't need it yet, and it creates various problems:
// 1. you'd need to store the current size in the shared memory area itself
// 2. If you have a big shared memory area, you have to start worrying about how
// to make sure it gets freed once the application (i.e. php) quits.
// I have a feeling problems like this is have caused problems in ASP (where I've
// used the 'Application' object to store megs of data over time and had it
// degrade to the point of standstill on me).
// Bear in mind the program dies if the memory gets exceeded...
define("SHM_SIZE", 62*1024);
define("SHM_DB", "DB_NAME");
define("SHM_DB_TABLE", "DB_TABLE");
// This function sets a key
function set_cache($key, $value)
{
$cache_hash = get_cache_raw();
$cache_hash[$key] = $value;
$cache = serialize($cache_hash);
// We can assume the shared memory has been created by the get_cache_raw() operation above so...
$shm_id = shmop_open(SHM_IDENTIFIER, "w", 0644, SHM_SIZE);
// Use a semaphore to make sure we're the only task accessing it
$sem_id = sem_get(SHM_IDENTIFIER);
sem_acquire($sem_id);
shmop_write($shm_id, $cache, 0);
sem_release($sem_id);
// Similarly, we can assume the database is properly initialised...
$link = get_db_connection(SHM_DB);
$sql = "UPDATE ".SHM_DB_TABLE." SET settings = '".addslashes($cache)."'";
$result = mysql_query($sql, $link);
if (!$result)
exit("Could not update the database: $sql");
}
// This function gets a key
function get_cache($key)
{
$cache_hash = get_cache_raw();
return $cache_hash[$key];
}
// This function returns a local read-only copy of the hash.
function get_cache_raw()
{
$shm_id = @shmop_open(SHM_IDENTIFIER, "n", 0644, SHM_SIZE);
$cache_hash = array();
// If we've created a new one, initialise it from the db
if ($shm_id) {
$link = get_db_connection(SHM_DB);
$result = mysql_query("SELECT settings FROM ".SHM_DB_TABLE, $link);
if (mysql_num_rows($result) > 0) {
$row = mysql_fetch_row($result);
$cache_hash = unserialize(stripslashes($row[0]));
} else {
// If there's nothing in the db, we have to initialise the db too
$sql = "INSERT INTO ".SHM_DB_TABLE." (settings) VALUES (".addslashes(serialize($cache_hash)).")";
$result = mysql_query($sql, $link);
if (!$result)
exit("Couldn't initialise the database with the cache: $sql");
}
// IMPORTANT: if the hash is too big for the memory, the program dies.
if (strlen(serialize($cache_hash)) > SHM_SIZE)
exit("The cache was too big for the available shared memory size.");
$sem_id = sem_get(SHM_IDENTIFIER);
sem_acquire($sem_id);
$result = shmop_write($shm_id, serialize($cache_hash), 0);
sem_release($sem_id);
} else {
$shm_id = shmop_open(SHM_IDENTIFIER, "a", 0, 0);
$cache_hash = unserialize(shmop_read($shm_id, 0, SHM_SIZE));
}
return $cache_hash;
}
?>
|
|
Unsing SHM as a general data storage medium
|
Ulf Wendel
|
20.04.2001 18:52
|
|
Hi,
first of all thanks for the introductionary article, keep on writing.
Inpired by the speed of SHM (what can be faster but RAM?) I started to write a SHM container for the PEAR Cache but got really frustrated. Dealing with one, two or 10 SHM segments is quite easy and may
speed up your applications dramatically, but an implementation of a data storage container is really hard.
Session ID usually have a length of 32 Bytes, the PEAR Cache ID uses a combination of a 32 Byte md5() hash plus a userdefined 127 Byte long group identifier as the primary key. That means one can not use the PK as a SHM key, you have eigther to skrink down your key from 32 (64|159) Bytes to 8 Byte SHM allows or you need an extra mapping (PHP array) from your PK to a SHM key.
The skrink means that you'll loose informations and the possibility of a clash caused by your hash function is extremly high compared to the risk of clashes when relaying on using md5()'s 32 Bytes. So this is no way you should take.
What you need is a mapping from your PK to a SHM key. This one has to be stored into SHM as well. Means you'll get the following memory layout:
[0x...0] Size of your Mapping
[0x...1] Your PK => SHM Key Mapping
[0x.2-n] Session/Cache entries
Ok, lets simulate a read access in this scenario. Your class gets the task to
return the value of the Session-ID/Cache-ID '1...'. First thing we have to do is to lock the SHM Key Mapping Segment [0x...1], read the content, unserialize it (this is what shm_put_var() hides from you), figure out if there's the requested Session-ID/Cache-ID, if so unlock the SHM Segment of the Key Mapping and finally read the content. Puuh quite a lot of overhead that eats up the SHM speed.
Lets inspire an insert now. First thing we have to do is to lock the SHM Key Mapping Segment [0x...1], read the content, unserialize it (this is what shm_put_var() hides from you), compute a SHM key for the new Session/Cache entry, add the key to the mapping, write the Session/Cache entry, serialize the mapping, save it back and finally unlock the Mapping Segment. Even worse than reading, but wait for a delete.
You can imagine that delete means nearly the same as insert with one major difference. You released a SHM key within a certain range of keys your Cache/Session is allowed to use and you have to make sure that you reuse the SHM key... Finally you'll end with a little memory manager :/.
Any suggestions how to solve the problem without writing a server in C that can...
Thx,
Ulf
|
|
RE: Unsing SHM as a general data storage medium
|
Rick Morris
|
23.07.2001 23:54
|
|
> Inpired by the speed of SHM (what can be faster but RAM?) I
...
> Finally you'll end with a little memory manager :/.
I appreciate the elegance of what you are trying to do, but what is the major benefit of this approach over using a RAM-resident SQL table (type=HEAP), or redirecting your Session file storage to a RAMdisk? I'm more interested in functionality than in performance. What are the possible functionality benefits of your shared memory manager over the other two approaches?
|
|
RE: Unsing SHM as a general data storage medium
|
neil davis
|
29.10.2001 02:10
|
|
> I appreciate the elegance of what you are trying to do, but
> what is the major benefit of this approach over using a
> RAM-resident SQL table (type=HEAP), or redirecting your
> Session file storage to a RAMdisk? I'm more interested in
> functionality than in performance. What are the possible
> functionality benefits of your shared memory manager over
Security is one advantage. If you don't want to run a database server, but want the benefits of a dynamic site, and only need read only access of the data, there is a big advantage.
Of course you could always put your xml files in a RAMdisk : ). In my case the killer operation is xml_parse_into_struct, not reading the data.
My entire record assembly for the content portion when loading a page costs 39ms +\- 2.5ms on a PII (66 mhz memory bus speed) with a 270k file(lots of html content blocks for pages). The xml_parse_into_struct call is responsible for 35 of those, shmem read, 2.4 +\- .04ms. File system read approximately the same(once cached by disk controller).
I think I need to store it in shmem as the parsed structure vs. the xml file data... back to work for me ...
l8,
neilio
|
|
Application Scope Variables
|
Ronnie Sengupta
|
19.02.2001 16:14
|
|
This was an excellent article. I was looking for a way to use global variables like you can in ASP in PHP. Now I have it!!!
I hope we see a lot of scripts using this, so I can snag the code.
|
|
RE: Application Scope Variables
|
richard correia
|
04.02.2002 07:00
|
|
> This was an excellent article. I was looking for a way to
> use global variables like you can in ASP in PHP. Now I have
> it!!!
>
> I hope we see a lot of scripts using this, so I can snag the
> code.
>
>
Hi ..
Do you have some code sample ready ..using semaphore ..
thank you ..
richard@webpercept.com
Mumbai
|
|
|
|
|
