-
Notifications
You must be signed in to change notification settings - Fork 9
/
Copy pathReadMe.html
1140 lines (900 loc) · 41.1 KB
/
ReadMe.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<meta charset="utf-8">
<meta name="author" content="Philip Semanchuk">
<meta name="copyright" content="All contents © 2016 Philip Semanchuk">
<meta name="keywords" content="python sysv system v ipc semaphore shared memory message queue">
<title>System V IPC for Python - Semaphores, Shared Memory and Message Queues</title>
<style type="text/css">
dt {
font-family: monospace;
font-weight: bold;
padding-bottom: .33em;
margin-top: 1em;
}
span[lang] { font-style: italic; }
span.param {
font-family: monospace;
font-style: italic;
}
span.book_title {
text-decoration: underline;
}
pre { margin-left: 2em; }
li { margin-top: 1em; margin-bottom: 1em; }
fieldset, legend {
background-color: #d0d0a9;
}
/* This style is only present on the local version of the readme.
In the online version, the RSS feed is displayed. */
div.rss { display: none; }
</style>
</head>
<body>
<h2>System V IPC for Python - Semaphores, Shared Memory and Message Queues</h2>
<div class="rss">
<a href="rss.xml"><img src="/common/images/rss.png" width="28" height="28" alt=""></a>
<br><a href="rss.xml">RSS</a>
</div>
<p>This describes the <tt>sysv_ipc</tt> module which gives Python access
to System V inter-process semaphores, shared memory and message queues
on most (all?) *nix flavors. Examples include OS X, Linux, FreeBSD,
OpenSolaris 2008.11, and (maybe) AIX.
It might also work under Windows with a library like
<a href="http://www.cygwin.com/">Cygwin</a>.
</p>
<p>It works with Python 2.7 and 3.x.
It's released
under a <a href="http://creativecommons.org/licenses/BSD/">BSD license</a>.
</p>
<p>You can <strong>download
<a href="sysv_ipc-0.7.0.tar.gz">sysv_ipc version 0.7.0</a>
</strong>
(<a href="sysv_ipc-0.7.0.md5.txt">[md5 sum]</a>,
<a href="sysv_ipc-0.7.0.sha1.txt">[sha1 sum]</a>,
<a href="sysv_ipc-0.7.0.sha256.txt">[sha256 sum]</a>)
which contains the source code, setup.py, installation instructions and
<a href="#samples">sample code</a>. You can read about
<a href="history.html#current">all of the changes in this version</a>.
</p>
<p>
You might also want to read
about some <a href="#bugs">known bugs</a>.
</p>
<p>You might be interested in the very similar module
<a href="/philip/posix_ipc/"><tt>posix_ipc</tt></a>
which provides Python access to POSIX IPC primitives. POSIX IPC is a little
easier to use than SysV IPC, but not all operating systems support it
completely.
</p>
<h2>Module <tt>sysv_ipc</tt></h2>
<p>Jump to <a href="#semaphore">semaphores</a>,
<a href="#shared_memory">shared memory</a>, or
<a href="#message_queue">message queues</a>.</p>
<h3>Module Functions</h3>
<dl>
<dt>attach(id, [address = None, [flags = 0]])</dt>
<dd>Attaches the (existing) shared memory that has the given <tt>id</tt> and
returns a new SharedMemory object. See
<a href="#attach">SharedMemory.attach()</a> for details on the
<tt>address</tt> and <tt>flags</tt> parameters.
<p>This method is useful only under fairly unusual circumstances.
You probably don't need it.
</p>
</dd>
<dt>ftok(path, id, [silence_warning = False])</dt>
<dd>Calls <tt>ftok(path, id)</tt>. Note that
<a href="#ftok_weakness"><tt>ftok()</tt> has limitations</a>, and this
function will issue a warning to that effect unless
<tt>silence_warning</tt> is True.
</dd>
<dt>remove_semaphore(id)</dt>
<dd>Removes the semaphore with the given <tt>id</tt>.</dd>
<dt>remove_shared_memory(id)</dt>
<dd>Removes the shared memory with the given <tt>id</tt>.</dd>
<dt>remove_message_queue(id)</dt>
<dd>Removes the message queue with the given <tt>id</tt>.</dd>
</dl>
<h3>Module Constants</h3>
<dl>
<dt>IPC_CREAT, IPC_EXCL and IPC_CREX</dt>
<dd><tt>IPC_CREAT</tt> and <tt>IPC_EXCL</tt> are flags used when
creating IPC objects. They're
bitwise unique and can be ORed together. <tt>IPC_CREX</tt> is
shorthand for <tt>IPC_CREAT | IPC_EXCL</tt>.
<p>When passed to an IPC object's constructor, <tt>IPC_CREAT</tt> indicates
that you want to create a new object or open an existing one. If you want
the call to fail if an object with that key already exists, specify
the <tt>IPC_EXCL</tt> flag, too.
</p>
</dd>
<dt>IPC_PRIVATE</dt>
<dd>This is a special value that can be passed in place of a key. It implies that
the IPC object should be available only to the creating process or its
child processes (e.g. those created with <tt>fork()</tt>).
</dd>
<dt>KEY_MIN and KEY_MAX</dt>
<dd>Denote the range of keys that this module accepts. Your OS might
limit keys to a smaller range depending on the typedef of
<tt>key_t</tt>.
<p>Keys randomly generated by this module are in the range
<tt>1 ≤ key ≤ SHRT_MAX</tt>.
That's type-safe unless your OS has a very bizarre
definition of <tt>key_t</tt>.
</p>
</dd>
<dt>SEMAPHORE_VALUE_MAX</dt>
<dd>The maximum value of a semaphore.
</dd>
<dt>PAGE_SIZE</dt>
<dd>The operating system's memory page size, in bytes. It's probably a good
idea to make shared memory segments some multiple of this size.
</dd>
<dt>SEMAPHORE_TIMEOUT_SUPPORTED</dt>
<dd>True if the platform supports timed semaphore waits, False otherwise.</dd>
<dt>SHM_RDONLY</dt>
<dd>Pass this flag to <tt>SharedMemory.attach()</tt> to attach the segment
read-only.</dd>
<dt>SHM_RND</dt>
<dd>You probably don't need this, but it can be used when attaching shared
memory to force the address to be
rounded down to SHMLBA. See your system's man page for <tt>shmat()</tt>
for more information.
</dd>
<dt>SHM_HUGETLB, SHM_NORESERVE and SHM_REMAP</dt>
<dd>You probably don't need these. They're Linux-specific flags that can
be passed to the SharedMemory
constructor, or to the <tt>.attach()</tt> function in the case of
SHM_REMAP. See your system's man page for <tt>shmget()</tt>
and <tt>shmat()</tt> for more information.
</dd>
</dl>
<h3>Module Errors</h3>
<p>In addition to standard Python errors (e.g. <tt>ValueError</tt>),
this module raises custom errors. These errors cover
situations specific to IPC.
</p>
<dl>
<dt>Error</dt>
<dd>The base error class for all the custom errors in this module. This
error is occasionally raised on its own but you'll almost
always see a more specific error.
</dd>
<dt>InternalError</dt>
<dd>Indicates that something has gone very wrong in the module code. Please
report this to the maintainer.
</dd>
<dt>PermissionsError</dt>
<dd>Indicates that you've attempted something that the permissions on the
IPC object don't allow.
</dd>
<dt>ExistentialError</dt>
<dd>Indicates an error related to the existence or non-existence of
an IPC object.
</dd>
<dt>BusyError</dt>
<dd>Raised when a semaphore call to <tt>.P()</tt> or <tt>.Z()</tt> either times out
or would be forced to wait when its <tt>block</tt> attribute is False.
</dd>
<dt>NotAttachedError</dt>
<dd>Raised when a process attempts to read from or write to a shared memory
segment to which it is not attached.
</dd>
</dl>
<h3 id="semaphore">The Semaphore Class</h3>
<p>This is a handle to a semaphore.</p>
<h4>Methods</h4>
<dl>
<dt>Semaphore(key, [flags = 0, [mode = 0600, [initial_value = 0]]])</dt>
<dd>Creates a new semaphore or opens an existing one.
<p><span class="param">key</span> must be <tt>None</tt>,
<tt>IPC_PRIVATE</tt> or
an integer > <tt>KEY_MIN</tt> and ≤ <tt>KEY_MAX</tt>. If the key
is <tt>None</tt>, the module chooses a random unused key.
</p>
<p>The <span class="param">flags</span> specify whether you want to create a
new semaphore or open an existing one.
</p>
<ul>
<li>With <span class="param">flags</span> set to the <strong>default</strong> of <tt>0</tt>, the module attempts
to <strong>open an existing</strong> semaphore identified by <span class="param">key</span> and raises
a <tt>ExistentialError</tt> if that semaphore doesn't exist.
</li>
<li>With <span class="param">flags</span> set to <tt>IPC_CREAT</tt>, the module
<strong>opens</strong> the semaphore identified by
<span class="param">key</span> <strong>or creates</strong> a new
one if no such semaphore exists. Using <tt>IPC_CREAT</tt> by itself
is not recommended. (See <a href="#sem_init">Semaphore Initialization</a>.)
</li>
<li>With <span class="param">flags</span> set to
<tt>IPC_CREX</tt> (<tt>IPC_CREAT | IPC_EXCL</tt>),
the module
<strong>creates a new semaphore</strong> identified by <span class="param">key</span>. If a
semaphore with that key already exists, the call raises an
<tt>ExistentialError</tt>.
<strong>The <span class="param">initial_value</span> is ignored unless
both of these flags are specified</strong> or
if the semaphore is read-only.
</li>
</ul>
<p>When opening an existing semaphore, <span class="param">mode</span> is ignored.
</p>
</dd>
<dt>acquire([timeout = None, [delta = 1]])</dt>
<dd>Waits (conditionally) until the semaphore's value is > 0 and then
returns, decrementing the semaphore.
<p>The <span class="param">timeout</span> (which can be a float) specifies how
many seconds this call should wait, if at all.
</p>
<p>The semantics of the timeout <a href="#v0_3">changed a little in
version 0.3</a>.
</p>
<ul>
<li>A <span class="param">timeout</span> of None (the default)
implies no time limit. The call will not return until its wait
condition is satisfied.
</li>
<li>When <span class="param">timeout</span> is 0, the call
raises a <tt>BusyError</tt> if it can't immediately
acquire the semaphore. Since it will
return immediately if <em>not</em> asked to wait, this can be
thought of as "non-blocking" mode.
</li>
<li>When the <span class="param">timeout</span> is > 0, the call
will wait no longer than <span class="param">timeout</span>
seconds before either returning (having acquired the semaphore)
or raising a <tt>BusyError</tt>.
</li>
</ul>
<p>When the call returns, the semaphore's value decreases by
<span class="param">delta</span>
(or more precisely, <tt>abs(<span class="param">delta</span>)</tt>)
which defaults to 1.
</p>
<p>On platforms that don't support the <tt>semtimedop()</tt> API call,
all timeouts (including zero) are treated as infinite. The call
will not return until its wait condition is satisfied.
</p>
<p>Most platforms provide <tt>semtimedop()</tt>. OS X is a
notable exception. The module's Boolean constant
<tt>SEMAPHORE_TIMEOUT_SUPPORTED</tt>
is True on platforms that support <tt>semtimedop()</tt>.
</p>
</dd>
<dt>release([delta = 1])</dt>
<dd>
Releases (increments) the semaphore.
<p>The semaphore's value increases by <span class="param">delta</span>
(or more precisely, <tt>abs(<span class="param">delta</span>)</tt>)
which defaults to 1.
</p>
</dd>
<dt>P()</dt>
<dd>A synonym for <tt>.acquire()</tt> that takes the same parameters.
<p>"P" stands for
<span lang="nl">prolaag</span> or <span lang="nl">probeer te verlagen</span>
(try to decrease), the original name given by
<a href="http://en.wikipedia.org/wiki/Semaphore_(programming)">Edsger Dijkstra</a>.
</p>
</dd>
<dt>V()</dt>
<dd>A synonym for <tt>.release()</tt> that takes the same parameters.
<p>"V" stands for
<span lang="nl">verhoog</span> (increase), the original name given by
<a href="http://en.wikipedia.org/wiki/Semaphore_(programming)">Edsger Dijkstra</a>.
</p>
</dd>
<dt>Z([timeout = None])</dt>
<dd>Blocks until zee zemaphore is zero.
<p><span class="param">Timeout</span> has
the same meaning as described in <tt>.acquire()</tt>.
</p>
</dd>
<dt>remove()</dt>
<dd>
Removes (deletes) the semaphore from the system.
<p>As far as I can tell, the effect of deleting a semaphore that
other processes are still using is OS-dependent. Check your system's
man pages for <tt>semctl(IPC_RMID)</tt>.
</p>
</dd>
</dl>
<h4>Attributes</h4>
<dl>
<dt>key (read-only)</dt>
<dd>The key passed in the call to the constructor.</dd>
<dt>id (read-only)</dt>
<dd>The id assigned to this semaphore by the OS.</dd>
<dt>value</dt>
<dd>The integer value of the semaphore.</dd>
<dt>undo</dt>
<dd>Defaults to False.
<p>When True, operations that change the
semaphore's value will be undone (reversed) when
the process exits. Note that when a process exits, an undo operation
may imply that a semaphore's value should become negative or
exceed its maximum.
Behavior in this case is system-dependent, which means that
<strong>using this flag can make your code non-portable</strong>.
</p>
</dd>
<dt>block</dt>
<dd>
Defaults to True, which means that calls to <tt>acquire()</tt> and
<tt>release()</tt> will not return
until their wait conditions are satisfied.
<p>When False, these calls
will not block but will instead raise an error if they are unable
to return immediately.
</p>
</dd>
<dt>mode</dt>
<dd>The semaphore's permission bits.
<p>Tip: the following Python code will display
the mode in octal:<br>
<tt>print int(str(my_sem.mode), 8)</tt>
</p>
</dd>
<dt>uid</dt>
<dd>The semaphore's user id.</dd>
<dt>gid</dt>
<dd>The semaphore's group id.</dd>
<dt>cuid (read-only)</dt>
<dd>The semaphore creator's user id.</dd>
<dt>cgid (read-only)</dt>
<dd>The semaphore creator's group id.</dd>
<dt>last_pid (read-only)</dt>
<dd>The PID of the process that last called <tt>semop()</tt> (<tt>.P()</tt>,
<tt>.V()</tt> or <tt>.Z()</tt>) on this semaphore.
<p>Linux and OS X also set this when the semaphore's value is changed,
although doing so disagrees with the POSIX specification.
See <a href='https://bugzilla.kernel.org/show_bug.cgi?id=112271'>Linux kernel bug 112271</a>.
</p>
</dd>
<dt>waiting_for_nonzero (read-only)</dt>
<dd>The number of processes waiting for the value of the semaphore to become
non-zero (i.e. the number waiting in a call to <tt>.P()</tt>).
</dd>
<dt>waiting_for_zero (read-only)</dt>
<dd>The number of processes waiting for the value of the semaphore to become
zero (i.e. the number waiting in a call to <tt>.Z()</tt>).
</dd>
<dt>o_time (read-only)</dt>
<dd>The last time <tt>semop()</tt> (i.e. <tt>.P()</tt>, <tt>.V()</tt> or
<tt>.Z()</tt>) was called on this semaphore.
</dd>
</dl>
<h4>Context Manager Support</h4>
<p>These semaphores provide <tt>__enter__()</tt> and <tt>__exit__()</tt>
methods so they can be used in context managers. For instance --
</p>
<pre>
with sysv_ipc.Semaphore(name) as sem:
# Do something...
</pre>
<p>Entering the context acquires the semaphore, exiting the context releases
the semaphore. See <tt>demo4/child.py</tt> for a complete example.
</p>
<h3 id="shared_memory">The SharedMemory Class</h3>
<p>This is a handle to a shared memory segment.
</p>
<h4>Methods</h4>
<dl>
<dt>SharedMemory(key, [flags = 0, [mode = 0600, [size = 0 or PAGE_SIZE, [init_character = ' ']]]])</dt>
<dd>Creates a new shared memory segment or opens an existing one.
The memory is automatically attached.
<p><span class="param">key</span> must be <tt>None</tt>,
<tt>IPC_PRIVATE</tt> or
an integer > <tt>0</tt> and ≤ <tt>KEY_MAX</tt>. If the key
is <tt>None</tt>, the module chooses a random unused key.
</p>
<p>The <span class="param">flags</span> specify whether you want to create a
new shared memory segment or open an existing one.
</p>
<ul>
<li>With <span class="param">flags</span> set to the
<strong>default</strong> of <tt>0</tt>, the module attempts
to <strong>open an existing</strong> shared memory segment identified by
<span class="param">key</span> and raises
a <tt>ExistentialError</tt> if it doesn't exist.
</li>
<li>With <span class="param">flags</span> set to <strong><tt>IPC_CREAT</tt></strong>, the module
<strong>opens</strong> the shared memory segment identified
by <span class="param">key</span> <strong>or
creates</strong> a new one if no such segment exists.
Using <tt>IPC_CREAT</tt> by itself
is not recommended. (See <a href="#mem_init">Memory Initialization</a>.)
</li>
<li>With <span class="param">flags</span> set to
<strong><tt>IPC_CREX</tt></strong> (<tt>IPC_CREAT | IPC_EXCL</tt>),
the module
<strong>creates</strong> a new shared memory segment identified by
<span class="param">key</span>. If
a segment with that key already exists, the call raises
a <tt>ExistentialError</tt>.
<p>When both <tt>IPC_CREX</tt> is specified
and the caller has write permission, each byte in the new memory segment will be
initialized to the value of <span class="param">init_character</span>.
</p>
</li>
</ul>
<p>The value of <span class="param">size</span> depends on whether
one is opening an existing segment or creating a new one.
</p>
<ul>
<li>When opening an existing segment, <span class="param">size</span>
must be ≤ the existing segment's size. Zero is
always valid.
</li>
<li>When creating an new segment,
many (most? all?) operating systems insist on a <span class="param">size</span>
> <tt>0</tt>.
In addition, some round the size
up to the next multiple of PAGE_SIZE.
</li>
</ul>
<p>This module supplies a default
<span class="param">size</span> of <tt>PAGE_SIZE</tt> when
<tt>IPC_CREX</tt> is specified and <tt>0</tt> otherwise.
</p>
</dd>
<dt id="attach">attach([address = None, [flags = 0]])</dt>
<dd>
Attaches this process to the shared memory. The memory must be attached
before calling <tt>.read()</tt> or <tt>.write()</tt>. Note that the
constructor automatically attaches the memory
so you won't need to call this method unless you explicitly detach it
and then want to use it again.
<p>The address parameter allows one to specify (as a Python long) a memory
address at which to attach the segment. Passing None (the default)
is equivalent to passing NULL to <tt>shmat()</tt>. See that
function's man page for details.
</p>
<p>The flags are mostly only relevant if one specifies a specific address.
One exception is the flag <tt>SHM_RDONLY</tt> which, surprisingly,
attaches the segment read-only.
</p>
<p>Note that on some (and perhaps all) platforms, each call to <tt>.attach()</tt>
increments the system's "attached" count. Thus, if each call to
<tt>.attach()</tt> isn't paired with a call to <tt>.detach()</tt>,
the system's "attached" count for the shared memory segment will not
go to zero when the process exits. As a result, the shared memory
segment may not disappear even when its creator calls <tt>.remove()</tt>
and exits.
</p>
</dd>
<dt>detach()</dt>
<dd>Detaches this process from the shared memory.</dd>
<dt>read([byte_count = 0, [offset = 0]])</dt>
<dd>Reads up to <span class="param">byte_count</span> bytes from the
shared memory segment starting at <span class="param">offset</span>
and returns them as a bytes object (which is the same as a <tt>str</tt>
under Python 2).
<p>If <span class="param">byte_count</span> is zero (the default) the
entire buffer is returned.
</p>
<p>This method will never attempt to read past the end of the shared
memory segment, even when
<span class="param">offset</span> + <span class="param">byte_count</span>
exceeds the memory segment's size. In that case, the bytes
from <span class="param">offset</span> to the end of the segment are returned.
</p>
</dd>
<dt>write(some_bytes, [offset = 0])</dt>
<dd>Writes bytes (i.e. <tt>str</tt> in Python 2) to the shared memory,
starting at <span class="param">offset</span>. Passing a Unicode object
may work, but doing so is unsupported and may be explicitly deprecated
in a future version.
<p>If the offset + data would write outside of the segment,
this function raises <tt>ValueError</tt>.
</p>
<p>The bytes may contain embedded NULL bytes ('\0').
</dd>
<dt>remove()</dt>
<dd>Removes (destroys) the shared memory. Note that actual destruction of the
segment only occurs when all processes have detached.
</dd>
</dl>
<h4>Attributes</h4>
<dl>
<dt>key (read-only)</dt>
<dd>The key provided in the constructor.</dd>
<dt>id (read-only)</dt>
<dd>The id assigned to this segment by the OS.</dd>
<dt>size (read-only)</dt>
<dd>The size of the segment in bytes.</dd>
<dt>address (read-only)</dt>
<dd>The address of the segment as Python long.</dd>
<dt>attached (read-only)</dt>
<dd>If True, this segment is currently attached.</dd>
<dt>last_attach_time (read-only)</dt>
<dd>The last time a process attached this segment.</dd>
<dt>last_detach_time (read-only)</dt>
<dd>The last time a process detached this segment.</dd>
<dt>last_change_time (read-only)</dt>
<dd>The last time a process changed the uid, gid or mode on this segment.</dd>
<dt>creator_pid (read-only)</dt>
<dd>The PID of the process that created this segment.</dd>
<dt>last_pid (read-only)</dt>
<dd>The PID of the most last process to attach or detach this segment.</dd>
<dt>number_attached (read-only)</dt>
<dd>The number of processes attached to this segment.</dd>
<dt>uid</dt>
<dd>The segment's user id.</dd>
<dt>gid</dt>
<dd>The segment's group id.</dd>
<dt>mode</dt>
<dd>The shared memory's permission bits.
<p>Tip: the following Python code will display
the mode in octal:<br>
<tt>print int(str(my_mem.mode), 8)</tt>
</p>
</dd>
<dt>cuid (read-only)</dt>
<dd>The segment creator's user id.</dd>
<dt>cgid (read-only)</dt>
<dd>The segment creator's group id.</dd>
</dl>
<h3 id="message_queue">The MessageQueue Class</h3>
<p>This is a handle to a FIFO message queue.</p>
<h4>Methods</h4>
<dl>
<dt>MessageQueue(key, [flags = 0, [mode = 0600, [max_message_size = 2048]]])</dt>
<dd>Creates a new message queue or opens an existing one.
<p><span class="param">key</span> must be <tt>None</tt>,
<tt>IPC_PRIVATE</tt> or
an integer > <tt>0</tt> and ≤ <tt>KEY_MAX</tt>. If the key
is <tt>None</tt>, the module chooses a random unused key.
</p>
<p>The <span class="param">flags</span> specify whether you want to create a
new queue or open an existing one.
</p>
<ul>
<li>With <span class="param">flags</span> set to the
<strong>default</strong> of <tt>0</tt>, the module attempts
to <strong>open an existing</strong> message queue identified by
<span class="param">key</span> and raises
a <tt>ExistentialError</tt> if it doesn't exist.
</li>
<li>With <span class="param">flags</span> set to <strong><tt>IPC_CREAT</tt></strong>, the module
<strong>opens</strong> the message queue identified
by <span class="param">key</span> <strong>or
creates</strong> a new one if no such queue exists.
</li>
<li>With <span class="param">flags</span> set to
<strong><tt>IPC_CREX</tt></strong> (<tt>IPC_CREAT | IPC_EXCL</tt>),
the module
<strong>creates</strong> a new message queue identified by
<span class="param">key</span>. If
a queue with that key already exists, the call raises
a <tt>ExistentialError</tt>.
</li>
</ul>
<p>The <span class="param">max_message_size</span> can be increased
from the default, but be aware of the issues discussed in
<a href="#message_queue_limits">Message Queue Limits</a>.
</p>
</dd>
<dt>send(message, [block = True, [type = 1]])</dt>
<dd>Puts a message on the queue.
<p>The <span class="param">message</span> should be a bytes object
(a.k.a. <tt>str</tt> in Python 2) and can contain embedded
NULLs (ASCII <tt>0x00</tt>). Passing a Unicode object
may work, but doing so is unsupported and may be explicitly deprecated
in a future version.
</p>
<p>The <span class="param">block</span> flag specifies whether or
not the call should wait if the message can't be sent (if, for
example, the queue is full). When <span class="param">block</span>
is <tt>False</tt>, the call will raise a <tt>BusyError</tt> if
the message can't be sent immediately.
</p>
<p>The <span class="param">type</span> is
associated with the message and is relevant when calling
<tt>receive()</tt>. It must be > 0.
</p>
</dd>
<dt>receive([block = True, [type = 0]])</dt>
<dd>
Receives a message from the queue, returning a tuple of
<tt>(message, type)</tt>. The message is a bytes object
(a.k.a. <tt>str</tt> in Python 2).
<p>The <span class="param">block</span> flag specifies whether or
not the call should wait if there's no messages of the
specified type to retrieve. When <span class="param">block</span>
is <tt>False</tt>, the call will raise a <tt>BusyError</tt> if
a message can't be received immediately.
</p>
<p>The <span class="param">type</span> permits some control over
which messages are retrieved.
</p>
<ul>
<li>When <span class="param">type</span> <tt>== 0</tt>, the call
returns the first message on the queue regardless of its
type.
</li>
<li>When <span class="param">type</span> <tt>> 0</tt>, the call
returns the first message of that type.
</li>
<li>When <span class="param">type</span> <tt>< 0</tt>, the call
returns the first message of the lowest type that is ≤ the
absolute value of <span class="param">type</span>.
</li>
</ul>
</dd>
<dt>remove()</dt>
<dd>Removes (deletes) the message queue.</dd>
</dl>
<h4>Attributes</h4>
<dl>
<dt>key (read-only)</dt>
<dd>The key provided in the constructor.</dd>
<dt>id (read-only)</dt>
<dd>The id assigned to this queue by the OS.</dd>
<dt id="queue_max_size">max_size</dt>
<dd>The maximum size of the queue in bytes. Only a process with
"appropriate privileges" can increase this value, and on some
systems even that won't work. See
<a href="#message_queue_limits">Message Queue Limits</a> for details.
</dd>
<dt>last_send_time (read-only)</dt>
<dd>The last time a message was placed on the queue.</dd>
<dt>last_receive_time (read-only)</dt>
<dd>The last time a message was received from the queue.</dd>
<dt>last_change_time (read-only)</dt>
<dd>The last time a process changed the queue's attributes.
This does not seem to be implemented under OS X.</dd>
<dt>last_send_pid (read-only)</dt>
<dd>The id of the most recent process to send a message.</dd>
<dt>last_receive_pid (read-only)</dt>
<dd>The id of the most recent process to receive a message.</dd>
<dt>current_messages (read-only)</dt>
<dd>The number of messages currently in the queue.</dd>
<dt>uid</dt>
<dd>The queue's user id.</dd>
<dt>gid</dt>
<dd>The queue's group id.</dd>
<dt>mode</dt>
<dd>The queue's permission bits.
<p>Tip: the following Python code will display
the mode in octal:<br>
<tt>print int(str(my_mem.mode), 8)</tt>
</p>
</dd>
<dt>cuid (read-only)</dt>
<dd>The queue creator's user id.</dd>
<dt>cgid (read-only)</dt>
<dd>The queue creator's group id.</dd>
</dl>
<h3>Usage Tips</h3>
<h4>Sample Code</h4>
<p>This module comes with two demonstration apps. The first (in the
directory <tt>demo</tt>) shows how to use shared memory and semaphores.
The second (in the directory <tt>demo2</tt>) shows how to use
message queues.
<h4 id="ftok_weakness">The Weakness of <tt>ftok()</tt></h4>
<p>
Most System V IPC sample code recommends <tt>ftok()</tt> for generating an
integer key that's more-or-less random.
It does not, however, guarantee that the key it generates is unused. If
<tt>ftok()</tt> gives your application a key that some other application is
already using,
your app is in trouble unless it has a reliable second mechanism for generating
a key. And if that's the case, why not just abandon <tt>ftok()</tt> and use the
second mechanism exclusively?
</p>
<p>This is the weakness of <tt>ftok()</tt> -- it isn't guaranteed to give you
what you want. The <a href="http://www.unix.com/man-page/FreeBSD/3/ftok/">BSD
man page for <tt>ftok</tt></a> says it is "quite possible for the routine to
return duplicate keys". The term "quite possible" isn't quantified, but suppose
it means one-tenth of one percent. Who wants to have 1-in-1000 odds of a
catastrophic failure in their program, or even 1-in-10000?
</p>
<p>This module obviates the need for <tt>ftok()</tt> by generating random
keys for you. If your application can't use <tt>sysv_ipc</tt>'s automatically
generated keys because it needs to know the key in advance, hardcoding a
random number like 123456 in your app might be no worse than using
<tt>ftok()</tt> and has the advantage of not hiding its limitations.
</p>
<p>This module provides <tt>ftok()</tt> in case you want to experiment with it.
However, to emphasize its weakness, this version of <tt>ftok()</tt> raises a
warning with every call unless you explicitly pass a flag to silence it.
</p>
<p>This package also provides <tt>ftok_experiment.py</tt> so that you can observe
how often <tt>ftok()</tt> generates duplicate keys on your system.
</p>
<h4 id="sem_init">Semaphore Initialization</h4>
<p>When a System V sempahore is created at the C API level, the OS is not required
to initialize the semaphore's value. (This per
<a href="http://www.opengroup.org/onlinepubs/009695399/functions/semget.html">the
SUSv3 standard for <tt>semget()</tt></a>.)
Some (most? all?) operating systems initialize it to zero, but this behavior
is non-standard and therefore can't be relied upon.
</p>
<p>If sempahore creation happens in an predictable, orderly fashion, this isn't a
problem. But a
race condition arises when multiple processes vie to create/open the same semaphore. The
problem lies in the fact that when an application calls <tt>semget()</tt> with only
the <tt>IPC_CREAT</tt> flag, the caller can't tell whether or not he has
created a new semaphore or opened an existing one.
<strong>This makes it
difficult to create reliable code without using <tt>IPC_EXCL</tt>.</strong>
W. Richard Stevens' <span class='book_title'>Unix Network Programming Volume 2</span>
calls this "a fatal flaw in the design of System V semaphores" (p 284).
</p>
<p>
For instance, imagine processes P1 and P2. They're executing the same code,
and that code intends to share a binary semaphore.
Consider the following sequence of events at the startup of P1 and P2 –
</p>
<ol>
<li>P1 calls <tt>semget(IPC_CREAT)</tt> to create the semaphore S.</li>
<li>P2 calls <tt>semget(IPC_CREAT)</tt> to open S.</li>
<li>P1 initializes the semaphore's value to 1.</li>
<li>P1 calls <tt>acquire()</tt>, decrementing the value to 0.</li>
<li>P2, assuming S is a newly-created semaphore that needs to be initialized,
incorrectly sets the semaphore's value to 1.</li>
<li>P2 calls <tt>acquire()</tt>, decrementing the value to 0. Both processes
now think they own the lock.</li>
</ol>
<p>W. Richard Stevens' solution for this race condition is to check the value of
<tt>sem_otime</tt> (an element in the <tt>semid_ds</tt> struct that's
populated on the call to <tt>semctl(IPC_STAT)</tt> and which is exposed to
Python by this module) which
is initialized to zero when the semaphore is created and otherwise holds
the time of the last
call to <tt>semop()</tt> (which is called by <tt>P()</tt>/<tt>acquire()</tt>,
<tt>V()</tt>/<tt>release()</tt>, and <tt>Z()</tt>).
</p>
<p>In Python, each process would run something like this:
<pre>
try:
sem = sysv_ipc.Semaphore(42, sysv_ipc.IPC_CREX)
except sysv_ipc.ExistentialError:
# One of my peers created the semaphore already
sem = sysv_ipc.Semaphore(42)
# Waiting for that peer to do the first acquire or release
while not sem.o_time:
time.sleep(.1)
else:
# Initializing sem.o_time to nonzero value
sem.release()
# Now the semaphore is safe to use.
</pre>
<h4 id="mem_init">Shared Memory Initialization</h4>
<p>With shared memory,
using the <tt>IPC_CREAT</tt> flag without <tt>IPC_EXCL</tt>
is problematic <em>unless you know the size of the segment
you're potentially opening</em>.
</p>
<p>Why? Because when creating a new segment,
many (most? all?) operating systems demand a non-zero size. However,
when opening an existing segment, zero is the only guaranteed safe value
(again, assuming one doesn't know the size of the segment in advance).
Since <tt>IPC_CREAT</tt>
can open or create a segment, there's no safe value for the size under
this circumstance.
</p>
<p>As a (sort of) side note, the
<a href="http://www.opengroup.org/onlinepubs/009695399/functions/shmget.html">SUSv3
specification for <tt>shmget()</tt></a> says only that the size of a new
segment must not be less than "the system-imposed minimum". I
gather that at one time, some systems set the minimum at zero despite the
fact that it doesn't make much sense to create a zero-length shared memory
segment. I think most modern systems do the sensible thing and insist on
a minimum length of 1.
</p>
<h4 id="message_queue_limits">Message Queue Limits</h4>
<p>Python programmers can usually remain blissfully ignorant of memory
allocation issues. Unfortunately, a combination of factors makes them
relevant when dealing with System V message queues.