| |
|
 |
Inter-Process Communication in PHP
|
Luis returns with an article covering the real computer science stuff, how to use shared memory and semaphores.
Inter-Process Communication in PHP
Luis Argerich (Salutia.com)
Luis returns with an article covering the real computer science stuff, how to use shared memory and semaphores.
"Preoptimization is the root of all evil (Donald Knuth)"
Preparation
In this article I'm going to discuss the use of IPC (Inter-Process-Communication) mechanisms from PHP4, you'll need PHP 4.0.4 or better compiled with the following strings:
--enable-shmop --enable-sysvsem
|
The two extensions that we are going to use are the Shared Memory extension and the Semaphores extension, we are going to see what to do with each one later.
What is IPC
IPC (Inter-process communication) is a standard Unix mechanism to communicate processes in the same machine, basically IPC defines 3 different ways of communication: message queues, shared memory and semaphores. We are going to discuss shared memory and semaphores in this article.
Using Shared Memory Segments from PHP
The use of shared memory segments is a very good way to communicate between processes, basically you define a memory segment that can be shared between processes, if a process writes to the memory with some data the other processes can see it. In PHP you can for example control, for example the number of processes running, or put some data in memory to prevent all the php scripts from doing the same thing, generating the same result.
To create a shared memory segment you use:
$shm_id = shmop_open($key, $mode, $perm, $size); |
Where:
$key is a number to identify the shared memory segment, all the scripts wanting to access a given segment must know the key
$mode is the creation mode "c" is used to create a segment, "a" is used to access an existing segment
$perm defines the permissions to the segment
$size defines the size of the segment
The function returns an id which you have to use to read/write the segment (don't use it as a file, there are
special functions that we'll cover later)
Example:
$shm_id = shmop_open(0xff3, "c", 0644, 100); |
In this example we created a segment of 100 bytes.
If you want to access an existing segment you use shmop_open setting 0 as the third and fourth parameters.If you want to create or access a segment you proceed as if you were creating it.
What have I done?
There're some command line utilities in Unix to control IPC resources, you can use "ipcs" to check which resources are created and so
Example
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x00280267 0 root 644 1048576 3
0x00000000 1 nobody 600 46084 10 dest
0x00000000 2 nobody 600 46084 8 dest
0x00000ff3 131 nobody 644 100 0
------ Semaphore Arrays --------
key semid owner perms nsems status
0x00280269 0 root 666 14
------ Message Queues --------
key msqid owner perms used-bytes messages
|
As we can see we have 4 shared memory segments created, the last one (key 0x00000ff3) is the one that we created with our example. You can see some properties of IPC resources with IPCs
Going back
If you want to delete/remove a shared memory segment you can use the following PHP instruction:
The $id you have to use is the one that shmop_open returns when you create/access the segment
If you want to remove the segment from the command line you can use 'ipcrm shmid' for example 'ipcrm 131' (in our example) to delete the segment
WARNING: If you use ipcrm as root be very careful checking which segments you delete because you can destroy other processes' segments with unpredictable (almost always bad) results.
Comment List
| Topic: |
Author: |
Time: |
|
Here's a working example
|
Jez Humble
|
31.10.2002 19:08
|
|
<?
/**
* The following four functions implement a permanent
* hashtable. This hash has an application scope,
* i.e. its contents can be accessed from all php scripts
* running on this computer.
*
* Furthermore, it is cached in the db, i.e. it's
* inviolable. This is designed to keep configuration settings
* for the site. Its contents are buffered in memory, so there's
* no hit on the db when accessing the hash apart from the first
* time you read it, and of course when you write to it.
*
* php must be compiled with --enable-shmop and --enable-sysvsem
* Thanks to Luis Argerich for the idea:
* http://zez.org/article/articleview/46/1/
*
* NB you also need to set up the values to access the db
* in the function below, and create a table
* with a field called settings of type text.
* the name of the database and table should be inserted
* into the appropriate 'define's below (SHM_DB, SHM_DB_TABLE).
*/
function get_db_connection($db)
{
$link = mysql_connect("localhost", 'USERNAME', 'PASSWORD');
mysql_select_db($db, $link);
return $link;
}
define("SHM_IDENTIFIER", 0x8ea);
// Set size to max size of a mysql 'text' field (64k, minus a few to allow for slashes)
// It would be nice to be able to increase the size automatically to accommodate larger
// hashes, but I don't need it yet, and it creates various problems:
// 1. you'd need to store the current size in the shared memory area itself
// 2. If you have a big shared memory area, you have to start worrying about how
// to make sure it gets freed once the application (i.e. php) quits.
// I have a feeling problems like this is have caused problems in ASP (where I've
// used the 'Application' object to store megs of data over time and had it
// degrade to the point of standstill on me).
// Bear in mind the program dies if the memory gets exceeded...
define("SHM_SIZE", 62*1024);
define("SHM_DB", "DB_NAME");
define("SHM_DB_TABLE", "DB_TABLE");
// This function sets a key
function set_cache($key, $value)
{
$cache_hash = get_cache_raw();
$cache_hash[$key] = $value;
$cache = serialize($cache_hash);
// We can assume the shared memory has been created by the get_cache_raw() operation above so...
$shm_id = shmop_open(SHM_IDENTIFIER, "w", 0644, SHM_SIZE);
// Use a semaphore to make sure we're the only task accessing it
$sem_id = sem_get(SHM_IDENTIFIER);
sem_acquire($sem_id);
shmop_write($shm_id, $cache, 0);
sem_release($sem_id);
// Similarly, we can assume the database is properly initialised...
$link = get_db_connection(SHM_DB);
$sql = "UPDATE ".SHM_DB_TABLE." SET settings = '".addslashes($cache)."'";
$result = mysql_query($sql, $link);
if (!$result)
exit("Could not update the database: $sql");
}
// This function gets a key
function get_cache($key)
{
$cache_hash = get_cache_raw();
return $cache_hash[$key];
}
// This function returns a local read-only copy of the hash.
function get_cache_raw()
{
$shm_id = @shmop_open(SHM_IDENTIFIER, "n", 0644, SHM_SIZE);
$cache_hash = array();
// If we've created a new one, initialise it from the db
if ($shm_id) {
$link = get_db_connection(SHM_DB);
$result = mysql_query("SELECT settings FROM ".SHM_DB_TABLE, $link);
if (mysql_num_rows($result) > 0) {
$row = mysql_fetch_row($result);
$cache_hash = unserialize(stripslashes($row[0]));
} else {
// If there's nothing in the db, we have to initialise the db too
$sql = "INSERT INTO ".SHM_DB_TABLE." (settings) VALUES (".addslashes(serialize($cache_hash)).")";
$result = mysql_query($sql, $link);
if (!$result)
exit("Couldn't initialise the database with the cache: $sql");
}
// IMPORTANT: if the hash is too big for the memory, the program dies.
if (strlen(serialize($cache_hash)) > SHM_SIZE)
exit("The cache was too big for the available shared memory size.");
$sem_id = sem_get(SHM_IDENTIFIER);
sem_acquire($sem_id);
$result = shmop_write($shm_id, serialize($cache_hash), 0);
sem_release($sem_id);
} else {
$shm_id = shmop_open(SHM_IDENTIFIER, "a", 0, 0);
$cache_hash = unserialize(shmop_read($shm_id, 0, SHM_SIZE));
}
return $cache_hash;
}
?>
|
|
Unsing SHM as a general data storage medium
|
Ulf Wendel
|
20.04.2001 18:52
|
|
Hi,
first of all thanks for the introductionary article, keep on writing.
Inpired by the speed of SHM (what can be faster but RAM?) I started to write a SHM container for the PEAR Cache but got really frustrated. Dealing with one, two or 10 SHM segments is quite easy and may
speed up your applications dramatically, but an implementation of a data storage container is really hard.
Session ID usually have a length of 32 Bytes, the PEAR Cache ID uses a combination of a 32 Byte md5() hash plus a userdefined 127 Byte long group identifier as the primary key. That means one can not use the PK as a SHM key, you have eigther to skrink down your key from 32 (64|159) Bytes to 8 Byte SHM allows or you need an extra mapping (PHP array) from your PK to a SHM key.
The skrink means that you'll loose informations and the possibility of a clash caused by your hash function is extremly high compared to the risk of clashes when relaying on using md5()'s 32 Bytes. So this is no way you should take.
What you need is a mapping from your PK to a SHM key. This one has to be stored into SHM as well. Means you'll get the following memory layout:
[0x...0] Size of your Mapping
[0x...1] Your PK => SHM Key Mapping
[0x.2-n] Session/Cache entries
Ok, lets simulate a read access in this scenario. Your class gets the task to
return the value of the Session-ID/Cache-ID '1...'. First thing we have to do is to lock the SHM Key Mapping Segment [0x...1], read the content, unserialize it (this is what shm_put_var() hides from you), figure out if there's the requested Session-ID/Cache-ID, if so unlock the SHM Segment of the Key Mapping and finally read the content. Puuh quite a lot of overhead that eats up the SHM speed.
Lets inspire an insert now. First thing we have to do is to lock the SHM Key Mapping Segment [0x...1], read the content, unserialize it (this is what shm_put_var() hides from you), compute a SHM key for the new Session/Cache entry, add the key to the mapping, write the Session/Cache entry, serialize the mapping, save it back and finally unlock the Mapping Segment. Even worse than reading, but wait for a delete.
You can imagine that delete means nearly the same as insert with one major difference. You released a SHM key within a certain range of keys your Cache/Session is allowed to use and you have to make sure that you reuse the SHM key... Finally you'll end with a little memory manager :/.
Any suggestions how to solve the problem without writing a server in C that can...
Thx,
Ulf
|
|
RE: Unsing SHM as a general data storage medium
|
Rick Morris
|
23.07.2001 23:54
|
|
> Inpired by the speed of SHM (what can be faster but RAM?) I
...
> Finally you'll end with a little memory manager :/.
I appreciate the elegance of what you are trying to do, but what is the major benefit of this approach over using a RAM-resident SQL table (type=HEAP), or redirecting your Session file storage to a RAMdisk? I'm more interested in functionality than in performance. What are the possible functionality benefits of your shared memory manager over the other two approaches?
|
|
RE: Unsing SHM as a general data storage medium
|
neil davis
|
29.10.2001 02:10
|
|
> I appreciate the elegance of what you are trying to do, but
> what is the major benefit of this approach over using a
> RAM-resident SQL table (type=HEAP), or redirecting your
> Session file storage to a RAMdisk? I'm more interested in
> functionality than in performance. What are the possible
> functionality benefits of your shared memory manager over
Security is one advantage. If you don't want to run a database server, but want the benefits of a dynamic site, and only need read only access of the data, there is a big advantage.
Of course you could always put your xml files in a RAMdisk : ). In my case the killer operation is xml_parse_into_struct, not reading the data.
My entire record assembly for the content portion when loading a page costs 39ms +\- 2.5ms on a PII (66 mhz memory bus speed) with a 270k file(lots of html content blocks for pages). The xml_parse_into_struct call is responsible for 35 of those, shmem read, 2.4 +\- .04ms. File system read approximately the same(once cached by disk controller).
I think I need to store it in shmem as the parsed structure vs. the xml file data... back to work for me ...
l8,
neilio
|
|
Application Scope Variables
|
Ronnie Sengupta
|
19.02.2001 16:14
|
|
This was an excellent article. I was looking for a way to use global variables like you can in ASP in PHP. Now I have it!!!
I hope we see a lot of scripts using this, so I can snag the code.
|
|
RE: Application Scope Variables
|
richard correia
|
04.02.2002 07:00
|
|
> This was an excellent article. I was looking for a way to
> use global variables like you can in ASP in PHP. Now I have
> it!!!
>
> I hope we see a lot of scripts using this, so I can snag the
> code.
>
>
Hi ..
Do you have some code sample ready ..using semaphore ..
thank you ..
richard@webpercept.com
Mumbai
|
|
|
|
|
