implementing container classes in shared memory
TRANSCRIPT
IMPLEMENTING CONTAINER CLASSES IN SHARED
MEMORY
by
François Bergeon
A DISSERTATION
Submitted to
The University of Liverpool
in partial fulfillment of the requirements for the degree of
MASTER OF SCIENCE Information Technology (Software Engineering)
January 5, 2009
ABSTRACT
IMPLEMENTING CONTAINER CLASSES IN SHARED
MEMORY
by
François Bergeon
The subject of this dissertation is the implementation of container classes
in shared memory in order to allow concurrent high-level access to easily
retrievable structured data by more than one process.
The project extends the sequence containers and associative containers
offered by the C++ Standard Template Library (STL) in the form of a
“Shared Containers Library” which purpose is to allocate both the
containers and the objects they contain in shared memory to make them
accessible by other processes running on the same machine.
Shared STL-like containers can be used for a variety of implementations
that require high-level sharing of structured and easily retrievable data
between processes. Applications of shared containers may include real-
time monitoring, metrics gathering, dynamic configuration, distributed
processing, process prioritization, parallel cache seeding, etc.
The Shared Container Library presented here consists of a binary library
file, a suite of C++ header files, and the accompanying programmers’
documentation. STL-type containers can be allocated in a shared memory
segment thanks to a class factory mechanism that abstracts the
complexity of the implementation. The Shared Containers Factory offers
synchronization for concurrent access to containers in the form of a
mechanism offering both exclusive and a non-exclusive locks.
The target platform of the Shared Container Library is the Microsoft
implementation of the STL for the 32bit Microsoft Windows operating
systems (Win32) as distributed with the Microsoft Visual Studio 2008
development environment and its Microsoft Visual C++ compiler. Further
developments of this library may include porting to the Unix platform and
designing a similar library for other languages such as Java and C#.
DECLARATION
I hereby certify that this dissertation constitutes my own product, that
where the language of others is set forth, quotation marks so indicate, and
that appropriate credit is given where I have used the language, ideas,
expressions, or writings of another.
I declare that the dissertation describes original work that has not
previously been presented for the award of any other degree of any
institution.
Signed,
François Bergeon
ACKNOWLEDGEMENTS
The author wishes to thank Martha McCormick, the dissertation advisor for the project, for
her support, kind words and constructive comments.
The author thanks the members of the EFSnet development team for their insight and
support during the development of an online transaction processing system when the
idea of implementing containers in shared memory for monitoring and dynamic
configuration was born.
Thanks to Eddy Luten for his high-resolution timer code that is used in the test
scaffolding. Thanks to Gabriel Fleseriu and Andreas Masur for pointing out the adequacy
of segregated storage in custom allocators. Thanks to Ion Gaztañaga for his work on the
Boost.Interprocess library and to Dr. Marc Ronell for his prior work on shared allocators.
A special thanks to Roy Soltoff, formerly of Logical Systems Inc, who authored “LC”
(“elsie”), a simple C compiler for the TRS-80 with which the author learned the C
language in 1982.
vi
TABLE OF CONTENTS
Page
LIST OF TABLES ix
LIST OF FIGURES x
LIST OF CODE LISTINGS xi
Chapter 1. Introduction 1 1.1 Overview 1 1.2 Scope 1 1.3 Problem Statement 2 1.4 Approach 4 1.5 Outcome 4
Chapter 2. Background and review of literature 6 2.1 Related Work 6 2.2 Literature 6
Chapter 3. Theory 8 3.1 Generic Programming and the Standard Template Library 8 3.2 Interprocess communication and shared memory 8 3.3 Memory allocation and segregated storage 9 3.4 Object oriented approaches – design patterns 11
3.4.1 Inheritance ............................................................................................................11 3.4.2 Abstraction, encapsulation, separation of concerns.............................................11 3.4.3 Design patterns .....................................................................................................12
Chapter 4. Analysis and Design 13 4.1 Introduction 13 4.2 Functional requirements 14
4.2.1 Allocation of container objects in shared memory..............................................14 4.2.2 Allocation of container’s contents in shared memory.........................................14 4.2.3 Coherence of internal data structures between processes ...................................15 4.2.4 Container attachment from a non-creating process .............................................15 4.2.5 Reference safety ...................................................................................................16 4.2.6 Concurrent access safety ......................................................................................16
4.3 Non-functional requirements 16
vii
4.4 Approach 17 4.5 Detailed design 20
4.5.1 Overview ..............................................................................................................20 4.5.2 Low level use cases ..............................................................................................23 4.5.3 High-level classes – shared containers and class factories..................................25 4.5.4 Process flows ........................................................................................................28
Chapter 5. Methods and Realization 31 5.1 Methodology 31 5.2 Iterative design steps 31 5.3 Shared containers library source code 32
5.3.1 Naming conventions.............................................................................................32 5.3.2 Structure of library and header files.....................................................................32 5.3.3 Interface to shared memory – pointer consistency ..............................................33 5.3.4 Object instantiation and attachment – map of shared containers ........................35 5.3.5 Synchronization....................................................................................................37 5.3.6 Reference safety ...................................................................................................38
5.4 Verification and validation 38 5.4.1 Unit testing ...........................................................................................................38 5.4.2 Test scaffolding ....................................................................................................39
Chapter 6. Results and Evaluation 42 6.1 Functional evaluation 42 6.2 Performance evaluation 43 6.3 Concurrency issues 45
Chapter 7. Conclusions 47 7.1 Potential applications 47 7.2 Lessons Learned 48 7.3 Future Activity 49 7.4 Prospects for Further Work 50
REFRENCES CITED 51
Appendix A. Shared Container Library Programmers’ Manual (excerpts) 54 A.1 Introduction to the Shared Containers Library 54 A.2 Shared heap parameters 55 A.3 sync 56 A.4 shared_vector<T > 58 A.5 shared_stack<T, Sequence > 59 A.6 shared_map <Key, Data, Compare> 61 A.7 shared_hash_set <Key, Compare> 63 A.8 SharedVectorFactory<T > 64
viii
A.9 SharedStackFactory<T , Sequence> 65 A.10 SharedMapFactory map<Key, Data, Compare> 67 A.11 SharedHashSetFactory set<Key, Compare> 69
Appendix B. Shared Containers Library Source Code (Excerpts) 71 B.1 safe_counter.h 71 B.2 sync.h 72 B.3 sync.cpp 74 B.4 heap_parameters.h 79 B.5 shared_heap.h 79 B.6 shared_heap.cpp 81 B.7 shared_pool.h 93 B.8 shared_pool.cpp 94 B.9 shared_allocator.h 102 B.10 container_factories.h 105 B.11 shared_deque.h 113 B.12 shared_stack.h 114 B.13 shared_set.h 116 B.14 shared_hash_map.h 117
Appendix C. Performance Evaluation Results 120
Appendix D. Test Scaffolding Source Code 121 D.1 VectorTest.cpp 121 D.2 MapTest.cpp 123
ix
LIST OF TABLES
Page
Table 1: Functional requirements ................................................................... 14 Table 2: Performance comparison ................................................................. 44 Table 3: Raw performance evaluation results .............................................. 120
x
LIST OF FIGURES
Page
Figure 1: Top level use cases......................................................................... 13 Figure 2: Sample linked list............................................................................. 15 Figure 3: Layered design class diagram......................................................... 22 Figure 4: Low level use cases ........................................................................ 23 Figure 5: Shared container creation, attachment and release ....................... 24 Figure 6: Shared container classes and class factories................................. 25 Figure 7: Container factories class hierarchy ................................................. 27 Figure 8: High-level use cases ....................................................................... 28 Figure 9: State chart diagram for the synchronization mechanism................ 29 Figure 10: Sample shared container usage ................................................... 30 Figure 11: Header file inclusion hierarchy ...................................................... 34 Figure 12: Test scaffolding interactive flow .................................................... 40 Figure 13: Monitoring / configuring an application server............................... 47
xi
LIST OF CODE EXCERPTS
Page
Listing 1: The shared_object sub-class of shared_pool........................ 36 Listing 2: Declaration of the the shared container map.................................. 36 Listing 3: Shared heap header structure and the get_params method ....... 37 Listing 4: safe_counter.h................................................................................. 71 Listing 5: sync.h .............................................................................................. 74 Listing 6: sync.cpp .......................................................................................... 78 Listing 7: heap_parameters.h ......................................................................... 79 Listing 8: shared_heap.h ................................................................................ 81 Listing 9: shared_heap.cpp............................................................................. 92 Listing 10: shared_pool.h................................................................................ 94 Listing 11: shared_pool.cpp.......................................................................... 102 Listing 12: shared_allocator.h....................................................................... 104 Listing 13: container_factories.h................................................................... 113 Listing 14: shared_deque.h .......................................................................... 114 Listing 15: shared_stack.h ............................................................................ 115 Listing 16: shared_set.h................................................................................ 117 Listing 17: shared_hash_map.h.................................................................... 119 Listing 18: VectorTest.cpp ............................................................................ 123 Listing 19: MapTest.cpp................................................................................ 125
1
Chapter 1. INTRODUCTION
1.1 Overview
The aim of this project is to offer a practical programming interface that allows developers
to implement generic container classes in shared memory. Such “shared containers”
provide concurrent high-level access to easily retrievable structured data by more than
one process.
This project extends the sequence containers and associative containers offered by the
C++ Standard Template Library (STL) in the form of a “Shared Containers Library” which
enables programmers to allocate both the containers and the objects they contain in
shared memory in order to make them accessible by other processes running on the
same machine.
Container classes can be found in most object oriented programming languages such as
C++ (“vectors”, “deques”, “sets”, “maps”, etc.) Java (“Maps”, “Sets”, “Lists”, etc.) and C#
(“Hashtables”, “Queues”, “Stacks”, etc.). Well-known data structures and algorithms are
used to store and retrieve data with flexibility and optimal performance. Inter-process
communication, on the other hand, offers low-level mechanisms to share data among
processes. A gap has been identified between the high-level access offered by container
classes and the low-level nature or typical inter-process communication. By implementing
container classes in shared memory, the gap can be filled with a mechanism that makes
high-level structured data concurrently accessible by various processes.
1.2 Scope
The main focus of the project is to extend the sequence containers and associative
containers offered by the C++ Standard Template Library (STL) so that the containers
and the objects they contain are allocated in a shared memory segment in order to be
accessible by other processes. Care was taken to preserve the properties of the
container classes when implemented in shared memory. In particular, the aim was to
2
maintain the genericity of the containers and to ensure that the different retrieval
mechanisms (e.g. iterators) and sorting algorithms offered by the Standard Template
Library still remain transparently usable when containers are allocated in shared memory.
This project concentrated on the Microsoft implementation of the STL for the 32bit
Windows operating system platform as distributed with Microsoft Visual Studio 2008.
Implementation of container classes in other languages such as Java and C# may be
discussed but no specific research for these platforms has been included in the scope of
this project. Access to shared memory was concentrated on the primitives offered by the
Win32 platform and no effort was made to research other operating system
implementations.
The following functional requirements have specifically been excluded of this project and
no effort was be made to address them:
- Data encryption or persistence
- Data access security
- Support for shared memory segment re-sizing
- Garbage collection
1.3 Problem Statement
Container classes are “classes whose purpose is to contain other objects.” These
classes, parameterized “to contain any class of object” (SGI 2006), constitute a
cornerstone of “generic programming,” a programming paradigm popularized by the C++
Standard Template Library (Stroustrup, as quoted by Buchanan 2008).
Since their introduction in the C++ Standard Template Library, container classes have
gained in popularity and similar implementations can now be found in most object
oriented programming languages. C++ containers like “vectors”, “deques”, “sets” and
“maps”, Java collections such as “Maps”, “Sets” and “Lists” and C# collections such as
“Hashtables”, “Queues” and “Stacks” all use well-known data structures and algorithms to
3
efficiently store and retrieve data in memory with great flexibility and performance.
Furthermore, the generic and parameterized approach of these container classes fosters
code reuse by abstracting the complexity of retrieval and sorting algorithms into
standardized libraries (Baum & Becker 2000). The corresponding library classes that
implement these containers are widely used in a large number of applications, but these
well-known software tools have some intrinsic limitations.
By default, both data and indexing structures are meant to be stored in the contiguous
memory space of the running process. If it is possible for different threads of a same
process to access data in these containers provided thread-safe code and adequate
synchronization are used, none of the aforementioned libraries currently offer a
mechanism that allows these structures to be concurrently accessible by more than one
process.
Mechanisms offered by operating systems for inter-process communication, on the other
hand, tend to only offer a low-level access to data. Shared memory, for example, consists
of a finite segment of memory, mapped to a file, and accessible concurrently and
randomly by various processes. But this mechanism only supports a low-level access to a
common contiguous memory region and requires a somewhat large amount of non-trivial
work to be usable. Like in the case of a raw buffer, it is up to the developer to structure
the data stored in a shared memory segment so that various processes can efficiently
access it.
By coupling the usability, performance and code reuse benefits of container classes along
with the inter-process communication nature of shared memory, the aim of this project is
to offer developers a practical high-level access to easily retrievable data shared between
programs while abstracting the inherent complexity of the implementation. Embracing
code reuse and logical encapsulation, the approach selected is to extend the container
classes offered by the C++ Standard Template Library (STL) so that both the containers
and the objects they contain can be located in a segment of shared memory, and are
therefore made concurrently accessible to various processes.
4
1.4 Approach
The main focus of the project is the C++ programming language on a 32-bit Microsoft
Windows platform. This language and this platform have been chosen for the availability
of a standard set of container classes (the STL) and by the well-documented nature of the
shared memory primitives offered by the Windows operating system. In addition, the C++
language allows for a straightforward implementation of low-level operating system calls,
an option that may not be directly available with other object oriented languages if the
needed primitives are not exposed in their standard library – which is usually not the
case.
The implemented design follows a layered approach consisting of a low level interface
managing a segment of shared memory, a higher-level shared memory pool responsible
for allocation and memory management, and a highest level programming interface
composed of classes derived from STL containers and their corresponding class
factories. Good practices of object-oriented analysis and design are embraced in the form
of code reuse, and abstraction of complexity through encapsulation and separation of
concerns.
1.5 Outcome
This project offers a practical method to instantiate STL containers in shared memory on
the Win32 platform. The approach and design selected provide a class library that allows
programmers to instantiate and access STL-like shared containers with little knowledge or
concern about the complexity of the underlying architecture.
Functional evaluation demonstrated that containers can be created and populated by one
process and concurrently accessed and modified by others. Performance evaluation
showed that, although containers located in shared memory may present degradation in
performance when simple data types are being considered, the response time for
populating and accessing a container composed of complex data type items – such as
strings – is not significantly longer than for similar operations in the process’ main
5
memory. Although the code developed for this project lacks some important features such
as solid exception handling, strong runtime type safety and garbage collection, it can
serve as the basis of a wider-scoped effort to standardize allocation of STL containers in
a shared memory on the Windows 32bit operating system or on Unix or Linux platforms.
6
Chapter 2. BACKGROUND AND REVIEW OF LITERATURE
2.1 Related Work
Ion Gaztañaga has extended the Boost initative with the Boost.Interprocess library, a C++
class library that offers a suite of containers similar to the STL’s and that can be
instantiated in shared memory (Gaztañaga 2008). However, the aim of this project differs
from Gaztañaga’s approach and one could argue that modifying the code of the STL
classes or designing all new classes may be considered contrary to object orientation and
code reuse principles.
In a paper entitled “A C++ pooled, shared memory allocator for the Standard Template
Library,” Dr. Ronell proposes an architecture for sharing STL containers between
processes (Ronell 2003). After contacting the author – who maintains a SourceForge
project for a shared allocator – it appears that the initiative suffered from a roadblock in
the form of an impossibility for some versions of the GNU C++ compiler (GCC) to
adequately instantiate shareable objects with the “placement new” directive. Dr. Ronell
went as far as submitting a bug report to the GCC committee before putting the project on
indefinite hold pending resolution of the issue and due to conflicting work schedule
(Ronell 2006).
Other initiatives range from various flavors or STL custom allocators to libraries
implementing standalone data structures in shared memory with various degrees of
success. One project of interest is the STLshm library which published aim is to “create
and use STL containers in […] shared memory” (anonymous n.d.), but according to the
apparently rarely updated SourceForge website, the project seems to be stalled and no
contact is provided for its author or caretaker to verify its progress.
2.2 Literature
Numerous books on the C++ language exist and the most commonly referenced is
certainly “The C++ Programming Language” authored by its creator (Stroustrup 2000).
7
The standard documentation of the STL provides some basic insights as to container
classes and their usage (SGI 2006). Additional literature provides extensive coverage of
implementation details of the STL (e.q. Austern 1999).
In his book “Effective STL” Scott Meyers (2001) details some of the idiosyncrasies of the
STL. In particular, this reading gives an understanding of some important limitations
placed upon custom allocators.
A simple implementation of a custom allocator using “segregated storage” (Stroustrup
2000, p.570-573) is the subject of an article containing sample pseudo-code by Fleseriu &
Masur (2004).
Charles Petzold’s “Programming Windows” (1999) covers Win32 mechanisms that can be
used for synchronizing access to shared memory between concurrent processes such as
mutexes and semaphores.
Academic papers and public websites present a number of initiatives related to containers
implemented in shared memory. Given gthe potential usefulness of the subject, it comes
as no surprise that such an implementaton has been attempted before.
In addition to Dr. Ronell’s paper mentioned above (Ronell 2003), a 2003 C++ User’s
Group Journal article reprinted by DrDobb’s magazine offers a special-purpose alocator
to make STL containers available in shared memory (Ketema 2003). But one may argue
that the author only describes a sample high-level architecture that falls short on concrete
details for an implementation and lacks coverage of some functional requirements like, for
example, the issue of pointer compatibility between processes.
8
Chapter 3. THEORY
3.1 Generic Programming and the Standard Template Library
Containers are an integral part of the Standard Template Library and some may argue
that generic programming represent a defining feature of the C++ language. In fact, the
STL draws its origins from the Ada Generic Library that pioneered the concepts of
generic programming as described by David R. Musser and Alexander A. Stepanov:
“Generic programming centers around the idea of abstracting from concrete, efficient
algorithms to obtain generic algorithms that can be combined with different data
representations to produce a wide variety of useful software” (Musser & Stepanov, 1988).
The authors went on to create the Ada Generic Library (Musser & Stepanov 1989)
followed a few years later by the C++ Standard Template Library (Stepanov & Lee 1995),
which was in turn incorporated in the standard C++ library (Austern 1999).
It is a commonly accepted idea that the abstraction mechanisms offered by the STL
provide great flexibility for data storage and retrieval using parameterized container
classes. One example of such a beneficial abstraction mechanism of particular interest to
this project’s aim is the availability of custom allocators. Custom STL allocators were
“originally developed as an abstraction for memory models […] to ignore the distinction
between near and far pointers” (Meyers 2001). This same mechanism can be used to
develop a custom memory allocation mechanism that differs from the default heap
allocation offered by the STL. In addition to changing the memory allocation strategy, it
can also be used in to redirect memory allocation to a managed segment of shared
memory (Ketema 2003).
3.2 Interprocess communication and shared memory
Mechanisms are offered by operating systems to allow processes to communicate
between each-other. Built-in primitives exist, for example, to create and manage mutexes
and semaphores, giving access to flags and counters meant to be used for
synchronization purposes (Downey 2008). Other mechanisms exist to transfer data
9
between processes sequentially such as with named pipes on the Unix operating system.
Shared memory is a more flexible mechanism that can be used to “allow multiple
processes to share virtual memory space” (Gray 1997).
On the Microsoft Windows platform, shared memory had a humble start by relying at first
on a specific data segment declared in a Dynamic Linked Library (DLL). The programmer
could allocate character strings in that segment and access them from the various
processes having loaded the DLL (Petzold 1999). With the introduction of the Win32
Application Programming Interface (API) – first in Windows NT, then in Windows 95 –
“true” shared memory capability was added to the operating system in the form of
memory-mapped files: “Memory-mapped files (MMFs) offer a unique memory
management feature that allows applications to access files on disk in the same way they
access dynamic memory—through pointers” (Kath 1993).
To access a segment of shared memory, a target memory-mapped file is first created for
this very purpose. A Windows File-Mapping object is created or accessed by a call to
CreateFileMapping or OpenFileMapping with the target memory-mapped file
handle as a parameter. Then, using Win32 API’s MapViewOfFileEx function, a view
into the File-Mapping object can be mapped into the process address space.
3.3 Memory allocation and segregated storage
If it can be said that “memory management is one of the most fundamental areas of
computer programming” (Bartlett 2004), one can also argue that there is no memory
allocation strategy that addresses all aspects of performance and resource utilization
efficiency in one simple solution. One of the fundamental components of this project is
the implementation of a low-level shared memory pool able to satisfy the allocation needs
of a custom STL allocator. Given the diverse choice of available allocation algorithms, the
determination was made to implement a simple segregated storage memory pool to
manage a segment of shared memory.
10
Segregated storage revolves around a “first-fit” allocation of memory chunks of fixed size
rather than allocation of variable size, dynamic memory sub-segments. When the
memory pool is first initialized, every fixed-size chunk is available and points to the next
contiguous chunk of memory in a manner similar to a linked list.
When an allocation request is processed, the fist available chunk is considered and the
next available chunk becomes the first available chunk for the next request. If the
required allocation size is smaller than one allocation chunk, no effort is made to reuse
the unemployed memory. If the required size is larger than one chunk, on the other hand,
a search for contiguous chunks with a total size at least that of the required allocation
size is performed in the memory pool. If enough contiguous chunks are found, they are
removed from the available pool by having the previous free chunk point to the next free
chunk and the address of the first chunk is returned.
When previously allocated memory is released into the pool, the released allocation
chunks are inserted at the beginning of the free chunk list. This algorithm presents the
advantage of simplicity and rapid allocation and deallocaiton response time (Purdom et
al. 1970). But numerous allocation and deallocation of various size blocks tend to
fragment the memory pool and may lead to its inability to satisfy subsequent allocation
requests of more than one chunk. A hardened segregated storage algorithm would
therefore not be complete without an adequate “garbage collection” mechanism that
guarantees that released memory chunks are returned to the pool in the most efficient
way while preserving the highest ratio of contiguous free space. It is not, however, part of
the scope of this project to design such a garbage collection mechanism and the
segregated storage memory pool developed for the shared memory segment of this
project does not make any effort at optimizing subsequent allocations. For this reason,
chunk size has to be chosen with great care: too large a chunk size and a lot of memory
is wasted for the most frequent allocations, too small a chunk size and
allocation/deallocation of multiple-chunk buffers may eventually fail because of high
fragmentation.
11
3.4 Object oriented approaches – design patterns
Of particular interest of this project, the following principles of Object-Oriented
Programming have been embraced during the design and implementation phases:
3.4.1 Inheritance
According to Peter Frohlïch (2002) “inheritance is an incremental modification
mechanism that transforms an “ancestor” class into a “descendent” class by augmenting
it in various ways.” In the spirit of this definition, inheritance allows for the reuse of the
STL container code into derived classes tailored to be allocated in shared memory. In
addition, the multiple inheritance feature offered by the C++ language allows for each
STL container class to be derived into a shared version that also derives from a
synchronization class.
Generalization and specialization through object inheritance is also present in the
hierarchy of shared container class factories. During the normal iterative refactoring
process for this project, a number of commonalities have been identified between the
constructors of the various STL containers. If a one-to-one relationship exists between
the type of shared container and its corresponding class factory, containers nevertheless
fall into simple sequence containers, associative containers and hash-based containers.
The class factory hierarchy is built along these same lines.
3.4.2 Abstraction, encapsulation, separation of concerns
As defined by Bennett et al. (2006) “encapsulation is a kind of abstraction that […]
focuses on the external behaviour of something and ignores the internal details of how
the behaviour is produced.” One of the aims of this project is to mask the internal
complexity of allocating containers in shared memory. Abstraction is used to isolate the
inherent complexity of interfacing with shared memory from the programmer by
encapsulating complex tasks into the lowest levels of the implementation and exposing a
simplified interface in the higher levels.
12
To use a term coined by Edsger W. Dijkstra, separation of concerns was used during the
analysis phase to ensure that responsibilities of the different objects falls along the lines
corresponding to a logically layered approach (Dijkstra 1974).
3.4.3 Design patterns
“Each pattern describes a problem which occurs over and over again in our environment,
and then describes the core of the solution to that problem, in such a way that you can
use this solution a million times over, without ever doing it the same way twice”
(Alexander et al., 1977). With the observation that different people sometimes solved
similar problems in similar ways, some well-known creational design patterns have found
their way in the architecture defined for this project.
In particular, two well-known design patterns were used during the design phase of this
project, the singleton and the abstract class factory.
The singleton pattern is applied in order to ensure that a only one shared heap object is
instantiated so as to provide a single access point of access for all shared memory
allocators (Gamma et al. 2002, p.127). This behavior is required by the fact that STL
custom allocators are not permitted to have non-static members (Meyer 2001, p 54), and
that a single segment of shared memory is targeted by the Shared Containers Library.
The abstract class factory pattern was used to design the mechanism responsible for
instantiating shared containers or connecting to existing ones. Class factories are defined
as providing an interface for creating families of related objects while abstracting the
complexity related to object instantiation (Gamma et al. 2002, p.87). – Shared containers
factories used in the design fall into this definition by allowing creation, attachment and
release of shared containers.
13
Chapter 4. ANALYSIS AND DESIGN
4.1 Introduction
From a high level approach, implementing container classes in shared memory needs to
satisfy a finite number of functional requirements. As it is the case for non-shared
containers, a process should be able to create a shared container, access it and release
its resources when it is no longer needed. But, in the case of shared containers,
processes need also to attach to an existing container created by another process and
safely access it concurrently. Figure 1 details the top level use cases for an
implementation of a container object in shared memory.
Figure 1: Top level use cases
For various processes to create new container objects, attach to an existing ones, access
and modify them concurrently, and then release their resources, the shared container
objects needs to be located in a segment of memory that is commonly accessible
between processes. In order to locate a container object in a segment of shared memory,
not only the object itself needs to be residing in shared memory, but all the objects it
contains along with its internal data structures have to be consistently accessible and
modifiable from all processes as well.
14
4.2 Functional requirements
At a lower level, the requirement of creating, attaching to and accessing a shared
container can be divided into items related to allocation of the container and items that
provide access to it (table 1).
Allocation requirements
- Allocation of container objects in shared memory
- Allocation of container’s contents in shared memory
- Coherence of internal data structures between processes
Access requirements
- Container attachment from a non-creating process
- Reference safety
- Concurrent access safety
Table 1: Functional requirements
4.2.1 Allocation of container objects in shared memory
For a container object to be accessible by more than one process, it needs to be located
inside a segment of shared memory. A mechanism is therefore needed to allocate a
buffer of shared memory and build a container object in this buffer. Considering a lower
level of requirements, it becomes clear that a mechanism is required to create and access
a segment of shared memory and manage variable size memory allocation within this
segment.
4.2.2 Allocation of container’s contents in shared memory
Along its lifespan, a container object allocates and deallocates memory for objects it
contains as they are inserted or removed. It also allocates memory for its own internal
data strucutres and storage needs. A mechanism is therefore needed to allow the
15
container object to interface with the above mentioned mechanism in order to allocate
variable size buffers in shared memory as needed.
4.2.3 Coherence of internal data structures between processes
It is the author’s contention that the internal data structure coherence requirement is less
trivial than the other functional requirements and may easily be overseen when first
considering this project. The inherent complexity of this requiement resides in ensuring
proper function of container objects used by concurrent processes. Specifically, the
internal data structures of shared containers need to consistently point to the valid
memory locations regardless of the process in which they are accessed.
To illustrate the discussion, consider the case of a simple linked list where each node
contains a pointer to the next element (figure 2). If the nodes are located in shared
memory, the pointers they contain need to consistently address the same memory
locations – regardless of the process in which they are accessed – or the linked list is
invalid. Without a mechanism devised to address this issue, a linked list built by process
“A” would contain pointers to elements which are valid in process “A” but point to invalid
memory addresses in process “B”.
Figure 2: Sample linked list
4.2.4 Container attachment from a non-creating process
Creating a container object located with its contents in shared memory and providing a
mechanism that ensures consistency of its internal data structures between concurrent
16
processes would be useless without a way for a process to gain access to a container
object created by another process. The requirement of container attachment from a non-
creating process ensures that a process that did not create a shared container can
nevertheless obtain access to it.
4.2.5 Reference safety
Once a process no longer has the need to access a shared container object, it is
desirable to release memory allocated to that object. But other processes may still require
access to it, and it cannot be destroyed without first verifying that it is no longer in use. A
reference tracking mechanism is required to identify the last process that uses a particular
shared container in order to destroy the container and release the memory it occupies
once that process no longer needs it.
4.2.6 Concurrent access safety
Two or more processes could be granted read access to a shared container (without
attempting to modify its contents) without penalty, but if a process attempts to access the
contents of a shared container while another one modifies it, the internal data structures
of the container may be in an intermediary state that would lead to unexpected results or
access violations. An access mechanism is therefore needed that allows any number of
concurrent reader processes, but ensures exclusive access to a process before it is
allowed to modify a shared container. This mechanism should guarantee that the writing
process is the only one accessing the container and that no other process is accessing
the same container at the time.
4.3 Non-functional requirements
No limitation should be imposed on the shared memory segment in terms of number of
shared containers it holds, overall size or individual size of containers, or number of
concurrent client processes.
17
The programming interface presented to users/developers should be consistent with that
of the STL and laid out in a clear, concise and logical way. Care should be taken to mask
undue complexity and favor practicality whenever possible.
4.4 Approach
To satisfy the requirements set forth above, the necessity of implementing a mechanism
to manage a memory pool in shared memory was considered. The aim of this memory
management mechanism is multiple. First, it should create and map a segment of shared
memory to the calling process address space and facilitate consistent access among
processes. Secondly, it is responsible for implementing a memory allocation scheme that
allows variable length buffers to be allocated, and to ensure that memory subsequently
released returns to the pool for possible reallocation. Finally, in its quality of focal point for
communication among the different processes, the memory management mechanism can
also be used to help satisfy the requirement of container attachment from a non-creating
process. In this regard, it can offer a centralized mechanism to link container objects to
unique identifiers that can be used by non-creating processes to obtain access to existing
containers.
Once a memory pool mechanism that allocates and manages a segment of shared
memory exists, creating a container object in shared memory can be performed by
invoking the “placement new” C++ instruction to instantiate the desired object in this
buffer. This language extension allows the “new” keyword to target a specific memory
address instead of allocating memory from the process heap (Stroustrup 2000, p.255-
256).
Note that, with the use of the placement new operator, final destruction of the shared
object cannot be performed by invoking the C++ “delete” instruction but rather should be
done first by calling the object’s destructor method and then by releasing the memory
buffer from the pool.
18
The functional requirement of allocating the container’s contents in shared memory is
addressed by using a mechanism provided by the STL in the form of a “custom allocator.
” A custom allocator is a class used as a parameter in the creation of a STL container
class that exposes allocation methods used by the container to allocate memory in a
custom location chosen by the programmer. Such a custom allocator can be designed to
use the above mentioned shared memory pool for its allocation need. It is outside the
scope of this discussion to compare the merits of different memory allocation strategies.
In this regards, the decision was made to use a form of segregated storage as
documented by Stroustrup (2000, pp.570-573).
The diverse responsibilities are given to the shared memory management mechanism
and points to a layered implementation in order to ensure a proper separation of
concerns. The lower level of this mechanism is in charge of interfacing directly with the
shared memory segment, whereas the higher level manages memory allocation within the
segment and facilitates low level communication between like-mechanisms in concurrent
processes. The highest level interfaces directly with the STL and offers a practical
programming interface.
The issue of consistency of the container’s internal data structures among processes is of
particular concern for the success of this project. After a superficial analysis, it would be
tempting to conclude that virtualization of memory references between processes can
simply be achieved by wrapping the container’s contents in relative pointers adjusted
within each process for the offset where the shared memory segment is mapped. If this
approach would indeed ensure consistency of pointers within each process’ address
space, this idea does not stand further scrutiny as it could solve the issue of virtualization
of the contents but it would not address the need to virtualize the container’s internal data
structures as illustrated by the linked-list example (figure 2).
In this regards, one could envision the need for a modification of the container classes so
that they use a relative pointer for their internal data structures as well. But this approach
requires a customized behavior on the part of the container classes that can only be
achieved with a modification of their source code. Changing the code of the Standard
19
Template Library clearly falls outside the scope of this project, and one may argue that it
goes against the core principles of reuse and encapsulation defined as some tenets of
object oriented programming. For these reason, other options were researched to satisfy
this requirement.
Although seemingly restrictive and apparently difficult to achieve, the approach selected
is in fact to ensure that the segment of shared memory is mapped at the very same offset
in each process. By forcing mapping of the shared memory segment to the same address
in each process space, each byte of shared memory is guaranteed to be located exactly
at the same offset in every process, and this solves the pointer consistency issue
altogether. This behavior guarantees consistency of the shared containers’ internal data
structures between the processes while at the same time reducing the complexity of the
implementation.
The requirement for concurrent access safety is satisfied by implementing a locking
mechanism. But, unlike the simplest existing forms of locking solutions, it was desirable to
design a mechanism that allowed multiple read-only processes to obtain a non-exclusive
lock ensuring that no writers were currently accessing the container, while allowing a
writer process to obtain an exclusive lock that ensure that no other writer and no readers
concurrently access the container.
To achieve the non-functional requirements of simplicity and accessibility of the
programming interface, the approach selected is to use a class factory mechanism that
can be invoked to create a new container, attach to an existing one or detach from a
container. The class factory approach masks the complexity of the implementation and
provides a high-level interface for accessing a shared container by a non-creating
process. It interfaces with the lower levels of the implementation to allocate memory in the
shared segment, obtain a pointer to a shared container object, and release unused
resources thanks to a built-in reference safety mechanism.
20
4.5 Detailed design
4.5.1 Overview
Problem domain analysis for the functional requirements set forth led to the selection of a
layered architecture approach. Interface to an allocated segment of shared memory is
layered between a low-level interface implemented by the shared_heap class and a
higher level interface implemented by the shared_pool class. The shared heap
manages the shared memory segment and generic allocation whereas the shared pool
manages allocation of objects and containers. The highest level of implementation is
materialized at the shared container factory level with the help of the
shared_allocator custom allocator class (figure 3).
The shared heap is implemented as a singleton class that offers a low-level interface to
an allocated segment of shared memory. The shared heap object implements the
mechanism for segregated storage allocation within the shared memory segment.
Other responsibilities of the shared heap include connecting client processes to the same
segment of shared memory while ensuring consistency between processes address
space and offering a way for the higher level interface to access common parameters. A
mechanism is also provided to detect when the last process releases the shared memory
segment in order to perform resource deallocation and cleanup.
The shared heap uses a discrete header allocated at the beginning of the shared memory
segment. This header allows the shared heap to map the shared memory segment to the
same address space in all processes in order to ensure pointer consistency between
processes. The header also holds thread safe variables used by the memory allocation
algorithm for synchronization, along with custom parameters common to all process and
used by higher level objects through its getparams method.
The shared pool is a higher level interface that implements access to the segment of
shared memory that is tailored to be used by shared containers. As a static class, it
enforces the singleton nature of the shared heap by forwarding allocation and
21
deallocation requests to a single static shared heap object. The shared_pool class
maintains a map of shared containers allocated in the segment of shared memory in
order to offer a mechanism for all class factories to create, attach to, and detach from
identifiable instances of shared containers. The shared map resides in shared memory
and is indexed by a unique container object name (string). It contains instances of
shared_object, a simple class defined to hold parameters and references to each
shared container. The shared pool uses the ptrparam parameter of the shared heap’s
header exposed through the getparams method to connect concurrent processes to the
shared container map.
The highest level of the interface consists of class factories deriving from a generalized
_SharedContainerFactory class. Class factories are used to instantiate and attach to
shared container object. A custom allocator class, shared_allocator, is used as a
template parameter for STL containers to target the segment of shared memory for their
allocation needs.
22
Figure 3: Layered design class diagram
23
4.5.2 Low level use cases
The different low level use cases of the system are depicted in figure 4.
The shared allocator is a STL custom allocator that uses the shared pool to interface to
the shared memory segment. Shared container factories also use the shared pool to build
new container objects in shared memory and insert newly created shared containers or
find existing ones in the container map that is kept in shared memory. The shared pool
keeps a reference counter updated for each shared container. When no more processes
hold any reference to a particular shared container, it is destroyed and the shared
memory it utilizes is freed by the class factory. The shared pool interface is then called
upon to remove the corresponding entry in the container map.
Figure 4: Low level use cases
The sequence diagram in figure 5 details a typical shared container creation, attachment
by another process and release.
24
Figure 5: Shared container creation, attachment and release
25
4.5.3 High-level classes – shared containers and class factories
4.5.3.1 Overview
Instantiation of shared containers is performed by using derived shared container classes
and their corresponding class factories.
For each container class defined in the STL, a corresponding shared container class and
a class factory are defined. The shared container class derives from the base STL
container class and from the sync class which provides a synchronization mechanism for
concurrent access. The corresponding class factory allows the programmer to create a
new shared container, attach to an existing one, or detach from a previously attached
instance. Each newly created shared container is identified with a user-assigned
alphanumeric name so that other processes can identify it and attach to it.
As an illustration, the shared_vector class derives from the STL vector class and
uses the SharedVectorFactory for instance creation and access. Similarly, the
shared_map class derives from the STL map class and is exposed through
SharedMapFactory (figure 6).
Figure 6: Shared container classes and class factories
26
The shared container factory classes have been generalized along the lines of the three
generic types of containers defined in the STL: sequential, associative and hash-based.
Each container type offers specific constructors that are exposed by the corresponding
class factories. Each shared container-specific class factory derives from the
corresponding type-generic container factory class. For example, as the vector class is
an STL sequence container, the SharedVectoryFactory class derives from the
_SharedSequenceContainerFactory class. This class, in turn, derives from the
generalized _SharedContainerFactory class. Figure 7 details the generalization and
specialization relationships between container factory classes. Common constructors are
implemented in the base _SharedContainerFactoryClass. A set of constructors
used by sequence containers are implemented in the derived
_SharedSequenceContainerFactory class. Another set of constructors for
associative containers are implemented in the
_SharedAssociativeContainerFactory class. Finally, a set of constructors specific
to hash-based containers are implemented in the _SharedHashContainerFactory
class. Container-specific class factories derive from one of these base classes according
to the type of container considered.
27
Figure 7: Container factories class hierarchy
28
4.5.3.2 High level use cases
The high level use cases of the system are depicted in figure 8.
Figure 8: High-level use cases
A client process can create a new shared container or attach to an existing one created
by another process. Before accessing the container it needs to ensure no other process is
modifying it by obtaining a non-exclusive read lock. Before modifying a shared container,
it needs to ensure that no other process is accessing it by obtaining an exclusive write
lock. The client process also needs to be able to release the locks. When its use of the
shared container is over, it releases it back, thereby signaling that no more access will be
performed on this object.
4.5.4 Process flows
Figure 9 is a state chart diagram detailing the different states and transitions of a sync
object.
29
Figure 9: State chart diagram for the synchronization mechanism
A process can obtain an exclusive or non-exclusive lock on the object, it can attempt to
upgrade an existing non-exclusive lock to an exclusive on, or it can downgrade an
exclusive lock into a non-exclusive one.
The following UML sequence diagram (figure 10) details a simple shared container
creation by one process and access by another one.
The first process creates the shared vector and obtains a write lock on it. The second
process attaches to the shared vector by name and attempts to get a read lock on it.
Once the first process releases the write lock, the second process obtains the read lock.
The first process releases the shared container. The second process releases the read
lock and releases the shared container. Because the second process was the last user,
the shared container is then destroyed and all shared memory occupied by it is released.
30
Figure 10: Sample shared container usage
31
Chapter 5. METHODS AND REALIZATION
5.1 Methodology
Due to the very nature of this project, an iterative approach similar to Boehm’s “spiral
model” was selected for analysis, design, implementation and testing (Boehm 1986). This
approach allowed for incremental discovery of the functional requirements as a better
understanding of the different issues was obtained. This approach proved valuable when
addressing the issue of pointer consistency, for example, and when the concurrency
issue linked to multi-core processors was unveiled.
5.2 Iterative design steps
Original analysis identified the functional requirements for this project and the issue of
pointer consistency among processes was researched in depth. In an attempt to address
the requirements in a most elegant way, a preliminary draft of the design evolved from a
class wrapper around STL containers to a parameterized class in the form:
shContainer<class T, class shAlloc>
where “T” was an STL container class and “shAlloc” was an implementation of a
custom STL allocator managing a segment of shared memory. With this approach, a
declaration for a shared vector of integers would have consisted of:
typedef shContainer<vector<int,shAlloc>,shAlloc> shINTVECTOR;
In the end, it was determined that this approach would have fallen short of implementing
some of the requirements (e.g. access by non-creating process) and may have proven to
be less intuitive to use by the programmer than the class factories.
The issue of pointer consistency among processes was researched and initial design
experimented with the concept of virtual pointers similar in concept to the auto_prt
STL class. Literature research showed that similar efforts to virtualize pointers in each
process were not being successful or required rewrite or modification of the STL
32
container classes and the decision was made to align the shared memory segment on the
same offset in all the processes.
5.3 Shared containers library source code
5.3.1 Naming conventions
The following naming conventions are used throughout the source code:
lowercase is used for most class names, attributes and operations, for
consistency with the STL.
BiCapitalisation is used for class factory names to differentiate them from
regular classes.
underscore_separated_words are used for class names and operations.
_leading_underscores are used for class attributes and for class names
meant to be derived (e.g. generalized class factories)
5.3.2 Structure of library and header files
The shared container library consists of a binary library file – shared_containers.lib
– and a series of C++ header files. First-line header files match corresponding STL
container header files and are meant to be included by programmers in their source code
(shared_vector.h, shared_map.h, shared_hash_set.h, etc). Each shared container class
is declared in its corresponding header file. Most header files include the corresponding
STL header and the container_factories.h header file. One exception is the
shared_stack and shared_queue classes that are declared in their respective
headers but include the shared_deque.h header file due to their adaptor nature.
Additional, lower-level header files are included by the container_factories.h
header file (figure 11). Due to the parameterized nature of the shared container classes,
most class instantiation logic is contained in the class_factories.h header files
rather than a compiled object library file. Lower-level interface to the shared memory pool
is implemented in the shared_pool.cpp and shared_heap.cpp files. The
33
synchronization class is implemented in the sync.cpp file. All cpp files are compiled into
the binary library file for distribution along with the headers. In addition to the library and
traditional header files, the heap_parameters.h header file contains a one-time
declaration of heap variables that need to be present at link time (listing 7). This header
can be included one time in any module of the C++ project. The external heap variables
can be customized at compile time and are used by the shared_heap object to open
and initialize the shared memory segment. These variables include the size and identifier
of the segment of shared memory, the size of memory chunks used by the segregated
storage algorithm and the target process’ base address for mapping the shared memory
segment.
5.3.3 Interface to shared memory – pointer consistency
To solve the issue of pointer consistency between processes it has been decided to map
the shared memory segment at the same offset in each process. Consistent mapping is
achieved by using the MapViewOfFileEx Win32 API calls that accepts a forced
mapping offset as an optional parameter and is available in Microsoft Windows 2000 and
newer operating systems, including Windows XP (Microsoft 2008 [1]).
To achieve consistent mapping among processes, the segment of shared memory is in
fact mapped twice. First the shared header is mapped at an address determined
automatically by the operating system so as not to conflcit with the rest of the process
memory. The shared header contains, among other things, the offset at which the rest of
the shared memory segment should be mapped in every process in order to ensure
pointer coherence. This value, stored in the _beaseaddress member of the shared
header (listing 3), is used for a second, separate mapping of the shared memory segment
which extends for the remainder of the segment size.
34
Figure 11: Header file inclusion hierarchy
35
Selecting an appropriate offset for mapping the segment of shared memory at the same
address in every process is non-trivial and no guarantee is made that mapping can be
obtained without errors. To facilitate consistent mapping, the offset should be high
enough in the process memory space that it does not interfere with the process heap, but
low enough so that it falls in the allowable addressing space of the process, while at the
same time providing enough “headroom” for a large enough segment of shared memory.
In Win32, a process memory space is limited to an addressable 2 Gb of memory for a
highest 32-bit address of 0x7fffffff (Kath 1993). Any value corresponding to the size
of the shared memory segment subtracted from the maximum address would be
appropriate as the starting offset of the mapped segment of shared memory. In the
current configuration, an arbitrary value of 0x01100000 has been selected for the 32-bit
mapping offset of the shared memory segment along with a heap size of 64 Mb or
0x04000000 bytes (listing 7). Base offset (0x01100000) + heap size (0x04000000) =
0x051000000, which is a lower memory address than the process’ highest address of
0x7fffffff.
5.3.4 Object instantiation and attachment – map of shared containers
The requirement of container attachment from a non-creating process is addressed by the
existence of a “master” list of shared containers allocated in the segment of shared
memory. This master list exists in the form of a STL map container that contains the
name of each shared container along with a structure containing a pointer to the shared
container object, its typeof size, and a reference counter. This structure is implemented in
the form of shared_object, a sub-class of the shared_pool class which is
responsible for managing the shared map (listing 1):
class shared_pool::shared_object { public: void* _object; // Pointer to object size_t _size; // Size of object safe_counter _refcount; // Usage count
36
shared_object(void* object, size_t size): _object(object), _size(size), _refcount(1) {};
shared_object() {}; ~shared_object() {}; };
Listing 1: The shared_object sub-class of shared_pool
The shared map is then defined as shared map of string and pointers to
shared_object objects which is allocated in the segment of shared memory and uses
the shared_allocator custom allocator for its allocation needs (listing 2).
template<class Key, class Data, class Compare=std::less<Key>> class shared_map : public sync, public std::map<Key, Data, Compare, shared_allocator<std::pair<const Key, Data>>> {};
typedef shared_map<std::string, shared_object*> container_map; static container_map* _containermap;
Listing 2: Declaration of the the shared container map
The mechanism implemented in order to grant access to the shared map from all
processes uses the get_params method exposed in the shared_heap class. Designed
to use an approach similar to the parameters used in Windows messaging (Petzold
1999), the get_params method returns a pointer to user-defined parameters stored in
the shared heap header (listing 3). The shared_pool class uses the _ptrparam
void* header member (returned in the ptrparam parameter) to store the address of the
shared map and make it accessible among processes.
__declspec(align(32)) volatile class shared_heap::shared_header { public: char _signature[sizeof(SHAREDHEAP_SIGNATURE)]; // Signature void* _baseaddress; // Common base address void* _head; // First available chunk size_t _heapsize, _chunksize; // Size of heap and chunk size long _longparam; // Client long parameter void* _ptrparam; // Client pointer parameter safe_counter _numprocesses; // Number of processes safe_counter _allocating; // 1 if allocation in progress char _filename[MAX_PATH]; // Temp filename }; void shared_heap::get_params(long** longparam, void*** ptrparam) { *longparam = &_header->_longparam;
37
*ptrparam = &_header->_ptrparam; }
Listing 3: Shared heap header structure and the get_params method
In addition, the shared_pool class performs simple verification when attaching to an
existing shared container. To this extent, the size of the container object returned by the
sizeof operator is validated against the size stored in the corresponding member in the
shared_object object in order to prevent mapping a pointer to a container of one type
to an existing shared container of a different type. This weak verification is far from
perfect and does not guarantee the denial of attachment of an incorrect shared container
type, but it is designed as a first line of defense in a more complex mechanism that can
be subsequently developed.
5.3.5 Synchronization
Interprocess synchronization is provided in the form of a low-level sync class from which
every shared container class derives using C++’s multiple inheritance mechanism (listings
5 & 6). The sync class uses safe_counter members that are designed as a wrapper
aroung a simple, thread-safe counter using Win32’s interlocked primitives (listing 4). Note
that the safe_counter type is also used in the shared heap’s header to facilitate
synchronization of allocation operations.
The sync class maintains a safe_counter for read locks and a separate one for write
locks. It provides a mechanism for obtaining a non-exclusive read lock or an exclusive
write lock, along with a mechanism for upgrading from a non-exclusive read lock to an
exclusive write lock, and for downgrading from an exclusive write lock to a non-exclusive
read lock. Downgrade is a guaranteed operation, but upgrade is not, and it does not
support an infinite timeout to avoid possible dead-lock scenarios (e.g. two threads
attempting to upgrade a non-exclusive lock simultaneously).
In addition, each sync object maintains a reference of the process and thread number
that owns an exclusive write lock in order to accept subsequent write lock and unlock
38
requests from the same thread/process without error. A fail-safe mechanism is provided
to prevent any lock counters to attain a negative value.
5.3.6 Reference safety
The shared_object class used in the shared container attachment mechanism
contains a safe_counter object used by the shared_pool for reference counting. The
counter is set to one when the shared container is created and incremented when a
process attaches to it. The counter is decremented each time a process detaches from
the container. When the counter reaches zero the container’s destructor is called and the
memory it occupied is released by the shared_pool.
5.4 Verification and validation
Due to the unavailability of an industry sponsor for this project, validation of the Shared
Container Library was limited to the author’s perception based on prior experience with
STL containers. Some key qualitative properties were confirmed during the
implementation of the test scaffolding software, in particular in terms of ease of
deployment and abstraction of complexity for the programmer-user. Unit testing was
performed on each component as an initial quality assurance step and a test scaffolding
program was designed to verify that the key functional requirements of the project were
met. Regression testing was performed with the help of the test scaffolding at each
incremental phase of the development effort.
5.4.1 Unit testing
Each component of the library was unit tested with an emphasis on structural coverage
and in particular branch coverage analysis. Due to the low-level nature of the produced
code, the decision was made to rely on the step-by-step feature of the development
environment’s built-in debugger to conduct most of the initial testing instead of using
repeatable unit testing tools such as “Cunit” which would have required an extensive
simulated running environment.
39
5.4.2 Test scaffolding
For the purpose of evaluating the functionalities and the performance of the produced
library, a test scaffolding program was developed to exercise a variety of STL containers
operations and types. The test scaffolding program was designed to be instantiated in
more than one process in order to test concurrency behavior.
In its interactive flow (figure 12), the test scaffolding first offers the choice to consider
containers allocated in the default process heap or in a shared memory segment. Then
the user is asked to choose what type of container is to be exercised among the
following:
- vector
- map
- queue
The user is then given a choice of operation to perform on the selected container:
- create/attach : create a new container or attach to an existing one
- populate : populate the container with discrete values
- verify : verify that discrete values populate the container
- destroy/detach : destroy the container or detach if still in use by another process
Each operation is timed with a millisecond timer in order to measure performance. A
typical functional evaluation consists in create, populate, verify and destroy operations.
Multi-process functional evaluations consists in create and populate operations in the
main process followed by concurrent attach, verify and detach operations in additional
processes.
Queue functional testing varies from other containers for populates makes one process
push values into the queue while verify makes another process pop and verify values
from the queue. Timeouts are adjusted for queue testing in order for the popping process
40
to wait for the pushing process without generating a verification error if the queue is
empty.
Figure 12: Test scaffolding interactive flow
In order to populate the containers with non-repetitive discrete values, the following
approach was used:
Non-repetitive numeric values are obtained by adding the order of the value to the
previous result to obtain a suite in the form “ni = n(i-1) + i” for i > 0 where n0 = 0
(listing 18).Similarly, non-repetitive string values are obtained by considering an ASCII
character value varying with the index number modulo 26, and building a string consisting
of the obtained character followed by a numeric representation of the index value (listing
19). The pseudocode form of this algorithm would be:
41
“Char(‘A’ + i%26) & i” for i ≥ 0
which would provides the following suite of values:
"A0", "B1", "C2", … , "Z25", "A26", "B27", etc.
42
Chapter 6. RESULTS AND EVALUATION
6.1 Functional evaluation
Functional evaluation of the shared containers was performed by instantiating a
container, populating it with non-repetitive but predictable discrete values and then
verifying the presence and correctness of the values in the container.
The test scaffolding was used for testing a vector-type container of integer values and a
map-type container of variable-length string keys and integer values. In each case, the
number of iteration was variable with final evaluation performed on 1,000,000 items.
Evaluation results were positive with the successful verification of the contents of a
container by one process while it had been populated by a different process, both for a
shared vector of integers and a shared map of string and integers.
The test scaffolding was also used to perform concurrent testing on a queue adapter to a
shared deque. One process “pushed” 10,000 integer values into the queue while another
process “popped” values from it. It was noted that push and pop operations on a shared
deque (or a stack or queue adapter) required exclusive access (write lock) due to the
fact that the first and last element of the deque were constantly being modified and
internal iterators were thus being invalidated.
The inefficiency of the segregated storage algorithm in terms of reallocation of multiple-
chunk buffers was evidenced by the original failure of a test consisting in creating and
populating a large map of strings and integers, destroying it, and then re-creating it. The
fact that the simple segregated storage algorithm does not offer any garbage collection
mechanism led to high fragmentation of the shared memory pool, and, more specifically,
to the deallocation of multiple chunks of memory in random order. After destroying the
map object, the shared pool consisted in free chunks of memory, few of which were
consolidated in the correct order with their immediate neighbors. This resulted in the
inability of the algorithm to allocate multiple-chunk buffers and led to a failure of the test
43
program. After analysis of the cause of the failure, it was decided to double the size of the
allocation chunk in the shared heap mechanism from its initial testing value of 32 bytes
thereby greatly reducing the number of multiple-chunk allocations (listing 7). This change
solved the reallocation error issue and multiple creation and destruction of large maps of
strings was subsequently successful.
6.2 Performance evaluation
For performance evaluation, the response time of standard operations on shared
containers was compared to the performance delivered by “pure” STL containers during
the same allocation, insertion of records, retrieval of records and release. Test was
performed allocating 1 million integer items into a vector and 1 million string and integer
pairs into a map. The comparison was made using the following parameters:
Local memory:
- Instantiation of pure STL vector and map containers
Shared memory with create:
- Instantiation of shared_vector and shared_map containers.
- First instantiation was not measured because it triggers creation and
initialization of the shared memory segment and associated segregated
storage pool
Shared memory with attach:
- Creation of shared_vector and shared_map containers by one process
followed by attach, populate, verify and release by another process, then
release of the container by the creating process for deallocation.
- No synchronization delays (successive, not concurrent access)
Table 2 details the results of this comparison.
44
Local memory Shared memory w/create
Shared memory w/attach
vector<int> - create / attach < 1ms < 1ms < 1ms - populate 13ms 18ms 17ms - verify 1ms 1ms 1ms - destroy / detach < 1ms < 1ms < 1ms map<string,int> - create / attach < 1ms < 1ms < 1ms - populate 1,699ms 1,683ms 1,752ms - verify 1,107ms 1,261ms 1,317ms - destroy / detach 393ms 193ms < 1ms
Table 2: Performance comparison
Local memory results were obtained by instantiating native STL classes. The “Shared
memory w/creation” column corresponds to standalone shared container classes. The
creation operation does not include initialization of the shared memory segment and its
corresponding segregated storage memory pool which was independently measured as
adding less than 100ms for a 64Mb shared pool with a chunk size of 64 bytes. The
“Shared memory w/attach” results were obtained by creating a container in one process
and then measuring attach, populate, verify and destroy operations from another
process. In each case, the tests have been run several times and the average results are
being presented here (see appendix C for complete results).
Creation time of the container objects in the “Shared memory w/creation” case is on par
with that of other cases. Response time for “attach” in the “Shared memory w/attach”
case reflects simple attachment to an existing shared container.
Once the shared memory segment is initialized, creation of a shared container is
performed in a similar time than the time required to create a local container. To this
duration, the time O(log n) required to insert a new container into the shared container
map’s balanced binary tree structure needs to be taken into account (where n is the
number of shared containers present in the container map). Similarly, the time required to
attach to an existing container is O(log n), the time required to retrieve the container by
name from the shared container map (Cormen et al. 2001).
45
One important finding is that, except when considering very simple data types, response
time to populate and verify a container in shared memory does not show significant
degradation compared to locally allocated containers. Performance related to populating
a container and verifying its contents does seem to suffer a 30-40% degradation when
simple data types are used (e.g. a vector of integers) but does not significantly differ for
complex types (e.g. a map of strings) whether the container is located on the process
heap or in shared memory. Of course, possible delays due to high disk activity or
concurrency locking requests are not taken into account in the above measured results.
Destruction of a shared memory allocated container proves to be faster than that of a
local one, a fact that may be explained by the fact that the segregated storage algorithm
used in this implementation does not make any attempt at garbage collection whereas
the default heap allocator may perform cleanup tasks when memory is deallocated. In
addition, the short time observed for detach in the “Shared memory w/attach” case
(<1ms) can be explained by the fact that the shared container is not actually erased as
the creating process still holds a reference to the object. In this case, the reference
counter of the shared container is simply decremented.
6.3 Concurrency issues
The postulate that “multithreaded programs are deterministic by nature […] many serious
bugs will not happen all the time, or even with any regularity” (Cohen & Woodring 1998)
was verified during the testing phase of this project.
Testing was conducted on two machines both running the same version of Windows XP
professional with the latest Microsoft updates. A desktop computer with a dual-core
Pentium 6400 processor running at 2.13 Ghz was used for development and testing, and
a laptop with a Pentium M processor at 1.6 Ghz was used for further testing and text
editing. Although concurrent multi-process testing on the laptop did not show any issue,
initial multi-process testing on the desktop computer with a debug version of the code
terminated with a fatal exception: “DEBUG_ERROR("ITERATOR LIST CORRUPTED!")”
46
The error was traced to the “xutility” header of the Dinkumware implementation of the
STL (based on code written by P.J. Plauger) distributed with Microsoft Visual C++ 2008.
Further research showed that the error was in fact linked to an iterator debugging feature
specific to this implementation of the STL. Because the error only occurred on the
desktop and not on the laptop, this issue may only exist on multi-core processors capable
of true multitasking as opposed to single core variants that only simulate multitasking by
the use of time-sharing techniques.
A solution was found in the form of defining the macros “_HAS_ITERATOR_DEBUGGING”
and “_SECURE_SCL” both with a value of zero to turn these additional features off at
compile time (Microsoft 2008 [2], Microsoft 2008 [3]). Once these macros were applied
and the test code recompiled, no additional faults were noted during subsequent multi
process concurrent testing on the desktop computer.
47
Chapter 7. CONCLUSIONS
7.1 Potential applications
Shared STL-like containers can be used for a variety of implementations that require
high-level access to structured data between processes.
Figure 13: Monitoring / configuring an application server
48
Such needs are frequently encountered in concurrent interactive applications where users
are allowed to exchange data. Other applications include monitoring of online transaction
processing system. Databases and local files can be used in lieu of shared containers for
these applications, but these solutions may add overhead, additional failure points and/or
increased code complexity compared to shared containers.
One practical example of shared container could be for a transaction processing system
where one could envision each worker thread of an application server updating a given
set of metrics residing in a shared map that can be read by another process. This other
process could, for example, expose these metrics via the Simple Network Management
(SNMP) protocol. The same SNMP interface could use another shared map to
dynamically modify the internal parameters representing the configuration used by the
worker threads, thereby providing a path to dynamic configuration (figure 13). One could
also envision a separate process using the metrics harvested by worker threads in the
shared container to dynamically change the configuration of the working system in order
to react to connection errors or system-wide congestion (e.g. automated database
failover, etc).
7.2 Lessons Learned
The subject of implementing STL containers in shared memory has attracted quite a bit of
attention over time, possibly because it may look easily feasible to the casual observer,
but proves to be in fact more challenging than originally thought. It would be an
oversimplification to think that implementing a custom allocator targeting a shared
memory segment would be sufficient to reach the goals of this project. In fact, allocating
the data held by the containers in shared memory is only one of several functional
requirements – and maybe one of the easiest to satisfy. Other requirements include
placing the internal indexing structures of the containers in shared memory and making
them consistently accessible by various processes. In addition, such an implementation
would not be complete without an adequate synchronization and reference counting
mechanism that guarantees safe concurrent access to the containers and their contents
49
but releases resources used by objects that are no longer in use. One could therefore
argue that the challenges presented by a project like this one may remain unrecognized
until one starts performing a detailed analysis of the implementation’s requirements.
Functional evaluation on the shared containers library led to a surprising finding in the
form of satisfactory performance on a computer equipped with a single core processor
but an unexpected behavior on a double-core equipped machine. The lesson here is that
multithreaded or multi-process applications should be tested on architectures where true
concurrency can occur – in this case a dual-core Pentium processor – to guarantee that
all potential synchronizing issues have been addressed. Processors with a high degree of
compatibility can still have significant architectural differences that may cause issues in
software programs to remain hidden for a long time.
7.3 Future Activity
The code implemented for this project lacks strong exception handling and this should be
added before the library is used in real-life applications. The simple segregated storage
algorithm does not currently offer a garbage collection mechanism and one could argue
that its usefulness would consequently be limited if numerous allocation/deallocation of
large memory buffers take place. A garbage collection mechanism would be a welcome
addition to the library.
In addition, the current library implementation only performs very limited runtime type
checking when attaching to an existing shared container. Only the memory footprint of
the container object is taken into account (its size returned by sizeof) to verify that the
container expected from a call to ContainerFactory::attach is of the correct type.
One could envision using C++’s runtime type information (RTTI) mechanism using the
typeid operator to verify the container’s type upon attachment by a non-creating
process.
50
Finally, no real-life testing of the shared container library has been performed at this time
and it would stand to reason to think that early practical usage of the library may unveil
issues that were not uncovered during initial testing.
7.4 Prospects for Further Work
The current implementation of the shared container library is limited to the Win32
operating system and the STL library distributed with Microsoft Visual Studio 2008.
Porting the shared containers library to other operating systems such as Unix or Linux
should prove feasible by developing platform-specific versions of the shared heap and the
safe counter classes. Developing a similar library for other languages such as Java and
C# would also be an interesting endeavor, albeit one that would pose its own set of
challenges and may lead to a different approach. For example, if parameterized classes
can be declared in newer versions of Java and C# by using “generics” similar in concept
and syntax to C++’s templates (Bracha 2004 & Microsoft 2008 [4]), neither languages
supports multiple inheritance which is extensively used in this implementation.
Another direction for further work could be to widen the features of the Shared Containers
Library. Additional desirable functionalities can be envisioned such as the possibility to
use more than one memory mapped file – a current limitation of this implementation – the
possibility to persist the memory mapped file on disk once all processes have terminated
execution and the addition of security and data encryption for the shared memory
segments and the containers they hold.
51
REFRENCES CITED
Alexander C. et al. (1977) - "A Pattern Language" - Oxford University Press - ISBN 978-0195019193 – p.x anonymous (n.d.) - "STLshm" - SourceForge.net - [Online] Available at: http://stlshm.sourceforge.net/Introduction.html (Accessed November 25, 2008) Austern, M. (1999) - "Generic Programming and the STL" - Addison Wesley - ISBN 0-201-30956-4 Bartlett, J. (November 16, 2004) - "Inside memory management - The choices, tradeoffs, and implementations of dynamic allocation" - IBM developerWorks - [Online] Available at: http://www.ibm.com/developerworks/linux/library/l-memory/ (Accessed December 14, 2008) Baum, L. & Becker, M. (2000) - "Generic components to foster reuse" - Proceedings of the 37th International Conference on Technology of Object-Oriented Languages and Systems - ISBN 0-7695-0918-5 – pp.266-277 Bennet S. et al. (2006) - "Object-Oriented Systems Analysis And Design Using UML" p.226 - McGraw Hill - ISBN 13-978-0-07-711000-0 Boehm, B.W. (May 5, 1986) - "A spiral model of software development and enhancement " - IEEE Computer - [Online] Available at: http://ieeexplore.ieee.org.ezproxy.liv.ac.uk/iel1/2/6/00000059.pdf?tp=&arnumber=59&isnumber=6 (Accessed December 18, 2008) Bracha G. (July 5, 2004) - "Generics in the Java Programming Language" - Sun Microsystems - [Online] Available: http://java.sun.com/j2se/1.5/pdf/generics-tutorial.pdf (Accessed December 18, 2008) Buchanan, J (March 27, 2008) - "An Interview with Bjarne Stroustrup" - Dr. Dobb's Journal - [Online] Available at: http://www.ddj.com/cpp/207000124 (Accessed November 17, 2008) Cohen, A. & Woodring, M. (1998) - "Win32 Multithreaded Programming" - O'Reilly - ISBN 1-56592-296-4 – p.570 Cormen, T. et al. (2001) - "Introduction to Algorithms" - McGraw Hill - ISBN 978-0-262-03293-3 – p.253 Dijkstra E.W. (1974) - "On the role of scientific thought - Selected writings on Computing: A Personal Perspective" - Springer-Verlag New York, Inc. - ISBN ISBN 0-387-90652-5 Downey, A.B. (2008) - "The Little Book of Semaphores" - Green Tea Press - [Online] Available at: http://greenteapress.com/semaphores/downey08semaphores.pdf (Accessed December 17, 2008) Fleseriu G. & Masur, A. (February 10, 2004) - "Allocators (STL)" - CodeGuru - [Online] Available at: http://www.codeguru.com/Cpp/Cpp/cpp_mfc/stl/article.php/c4079/ (Accessed October 13, 2008) Frohlïch, P.H. (2002) - "Inheritance Decomposed" - Department of Information and Computer Science University of California, Irvine - [Online] Available: http://www.cs.jyu.fi/~sakkinen/inhws/papers/Froehlich.pdf (Accessed December 10, 2008)
52
Gamma E. et al. (2002) - "Design Patterns - Elements of Reusable Object-Oriented Software" - Pearson Education - ISBN 81-7808-135-0 Gaztañaga , I. (March 28, 2008) - "Boost.Interprocess" – boost C++ libraries - [Online] Available at: https://svn.zib.de/lenne3d/lib/boost/1.35.0/doc/html/interprocess.html (Accessed November 25, 2008) Gray, J.S. (1997) - "Interprocess Communications in Unix" - Prentice Hall - ISBN 0-13-186891-8 – p.193 Kath, R. (February 9, 1993) - "Managing Memory-Mapped Files in Win32" - Microsoft Developer Network Technology Group - [Online] Available at: http://msdn.microsoft.com/en-us/library/ms810613.aspx (Accessed October 7, 2008) Ketema, G. (April 1, 2003) - "Creating STL Containers in Shared Memory" - Dr.Dobb's - [Online] Available at: http://www.ddj.com/cpp/184401639 (Accessed October 7, 2008) Meyers, S. (2001) - "Effective STL" - Addison Wesley - ISBN 0-201-74962-9 – pp.48-58 Microsoft (November 6, 2008 [1]) - "MapViewOfFileEx Function" - Microsoft Corp. - [Online] Available at: http://msdn.microsoft.com/en-us/library/aa366763(VS.85).aspx (Accessed November 25, 2008) Microsoft (2008 [2]) - "Debug Iterator Support" - Microsoft Corp. - [Online] Available at: http://msdn.microsoft.com/en-us/library/aa985982(VS.80).aspx (Accessed December 10, 2008) Microsoft (2008 [3]) - "Checked Iterators" - Microsoft Corp. - [Online] Available at: http://msdn.microsoft.com/en-us/library/aa985965(VS.80).aspx (Accessed December 10, 2008) Microsoft (2008 [4]) - "Introduction to Generics (C# Programming Guide" - Microsoft Corp. - [Online] Available at: http://msdn.microsoft.com/en-us/library/0x6a29h6(VS.80).aspx (Accessed December 18, 2008) Musser, D.R. & Stepanov A.A. (1988) - "Generic programming" - Proceeds of the First International Conference of ISSAC-88 and AAECC-6 - [Online] Available at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.51.1780&rep=rep1&type=pdf (Accessed December 8, 2008) Musser, D.R. & Stepanov A.A. (1989) - "The Ada Generic Library: Linear List Processing Packages” - Springer Compass International – ISBN: 0387971335
Petzold, C. (1999) - "Programming Windows" - Microsoft Press - ISBN 1-57231-995-X Purdom, P.W. et al. (August 1, 1970) - "Statistical Investigation of Three Storage Allocation Algorithms" - Computer Science Dept. - The University of Wisconsin Madison - [Online] Available at: http://www.cs.wisc.edu/techreports/1970/TR98.pdf (Accessed December 14, 2008) Ronell, M. (2003) - "A C++ Pooled, Shared Memory Allocator For The Standard Template Library" - Proceedings of the Fifth Real Time Linux Workshop - [Online] Available at: http://allocator.sourceforge.net/ (Accessed November 21, 2008) Ronell, M. (May 29, 2006) - "GCC Bugzilla Bug 21251 - Placement into shared memory" - GNU - [Online] Available at: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21251 (Accessed November 21, 2008)
53
SGI (2006) - "Standard Template Library Programmer's Guide - Introduction to the Standard Template Library" - Silicon Graphics, Inc. - Hewlett-Packard Company - [Online] Available at: http://www.sgi.com/tech/stl/stl_introduction.html (Accessed September 25, 2008) Stepanov A. & Lee M. (1995) - "The standard template library" - Hewlett-Packard Company - [Online] Available at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.108.7776&rep=rep1&type=pdf (Accessed December 8, 2008)
Stroustrup, B. (2000) - "The C++ Programming Language" - Addison Wesley - ISBN 0-201-70073-5
54
Appendix A. SHARED CONTAINER LIBRARY PROGRAMMERS’
MANUAL (EXCERPTS)
The latest version of the programmer’s documentation for the shared container library is available online at: http://bergeon.francois.perso.neuf.fr/containers/doc.html.
A.1 Introduction to the Shared Containers Library
The Shared Containers Library is a C++ library of container classes implemented in shared memory. Containers offered by the Shared Containers Library are derived from the containers offered in the Standard Template Library (STL). With the Shared Containers Library, programmers are able to instantiate STL containers and the objects they contain in a segment of shared memory accessible by concurrent processes. Programmers are responsible for using the provided synchronization mechanisms to avoid access and update conflicts between concurrent processes accessing a shared container. The Shared Containers Library was written by François Bergeon for his dissertation for a Master of Science in Information Technology (Software Engineering) at the University of Liverpool, UK (Martha McCormick Dissertation Advisor). Permission to use, copy, modify, and distribute this software and its documentation for any non-commercial purpose is hereby granted without fee, provided that the below copyright notice appears in all copies and that both the copyright notice and this permission notice appear in supporting documentation. Please contact the author at [email protected] to inquire about possible commercial use of this library. The author makes no representations about the suitability of this software for any purpose. It is provided "as is" without express or implied warranty. Copyright © 2008 François Bergeon
55
A.2 Shared heap parameters
Description
Shared heap parameters need to be defined in the code in order to be present at link time. Default values are defined in the header heap_paramters.h that can be included in the source code or modified to suit.
char* _heapname
_heapname is an arbitrary name assigned to the shared heap. The default value is “SHARED_CONTAINERS”.
size_t _heapsize
_heapsize is the size of the shared heap in bytes. The default value is 10,485,760 bytes or 10 Mbytes.
size_t _chunksize
_chunksize is the size of each allocation chunk in bytes. The default value is 32 bytes.
void* _baseaddress
_baseaddress is the address in the process address space where the shared heap is to be mapped. The default value is 0x01100000, an arbitrary high offset in the process space.
Definition
These values are defined in the header heap_parameters.h.
56
A.3 sync
Description
sync is a synchronization class designed to be used as-is or derived by classes that require synchronization. sync offers both exclusive (write) locks and non-exclusive (read) locks. Lock behavior follows the following matrix:
For A corresponding call to read_unlock is required for each call to read_lock. If the same thread calls write_lock more than once, the operation succeeds. A corresponding call to write_unlock is required for each successful call to write_lock. A call to upgrade_lock is functionally equivalent to a thread-safe call to read_unlock followed by a call to write_lock. Upgrade from a non-exclusive lock to an exclusive lock cannot be guaranteed. A call to downgrade_lock is functionally equivalent to a thread-safe call to write_unlock followed by a call to read_lock. Downgrade from an exclusive lock to non-exclusive lock is always guaranteed. A call to is_locking_thread returns true if the calling thread and process is the current owner of the exclusive lock. The timeout parameter is rounded down to the closest increment of 10ms. A value of less than 10 returns immediately. A value of -1 is equivalent to an infinite timeout. An infinite timeout should never be used in a call to upgrade due to the risk of a possible race condition.
Example
sync s; s.read_lock(); … if (s.upgrade_lock()) s.write_unlock(); else s.read_unlock();
Definition
Defined in the header sync.h.
Request\Existing lock none read lock write lock read_lock success success wait for release of write lock
write_lock success wait for release of all read locks wait for release of write lock
upgrade_lock n/a wait for release of all other read locks n/a
downgrade_lock n/a n/a success is_locking_thread false false true if thread owns the lock
57
Members
Member Description bool read_lock(long timeout = -1) Obtain non-exclusive read lock within timeout milliseconds,
0 for immediate or -1 for infinite. Returns true if lock obtained.
Void read_unlock() Release read lock bool write_lock(long timeout = -1) Obtain exclusive write lock within timeout milliseconds, 0
for immediate or -1 for infinite. Returns true if lock obtained.void write_unlock() Release write lock bool upgrade_lock(long timeout = 0)
Attempt to upgrade an existing non-exclusive read lock to an exclusive write lock within timeout milliseconds or 0 for immediate. Returns true if the lock was successfully upgraded. If the upgrade fails, the non-exclusive read lock is not released.
void downgrade_lock() Downgrade from an existing exclusive write lock to a non-exclusive read lock.
58
A.4 shared_vector<T >
Description
A shared_vector is a shared memory implementation of a STL vector container. shared_vector offers the same functionalities and properties than vector. Please refer to the STL documentation for information on the vector class. shared_vector also offers synchronization methods that should be used if other processes are susceptible to access it concurrently. shared_vector objects are be created, attached to and released from by calling the static methods of the SharedVectorFactory class.
Example
shared_vector<int> *V = SharedVectorFactory<int>::create(“name”); V->write_lock(); V->insert(V->begin(), 3); V->write_unlock(); V->read_lock(); assert(V->size() == 1 && V->capacity() >= 1 && *V[0] == 3); V->read_unlock();
Definition
Defined in the header shared_vector.h.
Template parameters
Parameter Description Default T The shared vector's value type: the type of object that is stored in the vector.
Type requirements
Same requirements than for vector.
Public base classes
sync
Members
All public members of the vector class and of the sync class are exposed.
59
A.5 shared_stack<T, Sequence >
Description
A shared_stack is a shared memory implementation of a STL stack adapter. A shared_stack offers the same functionalities and properties than the stack. Please refer to the STL documentation for information on the stack class. shared_stack also offers synchronization methods that should be used before accessing the container if other processes are susceptible to access it concurrently. shared_stack objects are created, attached to and released from by calling the static methods of the SharedStackFactory class.
Example
int main() { shared_stack<int> *S = SharedStackFactory<int>::create(“name”); S->write_lock(); S->push(8); S->push(7); S->push(4); assert(S->size() == 3); assert(S->top() == 4); S->pop(); assert(S->top() == 7); S->pop(); assert(S->top() == 8); S->pop(); assert(S->empty()); S->write_unlock(); SharedStackFactory<int>::detach(“name”); }
Definition
Defined in the header shared_stack.h.
Template parameters
Parameter Description Default T The shared deque's value type: the type of object that is stored
in the deque.
Sequence The type of the underlying container used to implement the stack.
shared_deque<T>
Type requirements
60
Same requirements than for stack.
Public base classes
sync
Members
All public members of the stack class and of the sync class are exposed.
61
A.6 shared_map <Key, Data, Compare>
Description
A shared_map is a shared memory implementation of a STL map container. A shared_map offers the same functionalities and properties than the map. Please refer to the STL documentation for information on the map class. shared_map also offers synchronization methods that should be used before accessing the container if other processes are susceptible to access it concurrently. shared_map objects are created, attached to and released from by calling the static methods of the SharedMapFactory class.
Example
struct ltstr { bool operator()(const char* s1, const char* s2) const { return strcmp(s1, s2) < 0; } }; int main() { shared_map<const char*, int, ltstr> *months = SharedMapFactory<const char*, int, ltstr>.create(“my map”); months->write_lock(); *months["january"] = 31; *months["february"] = 28; *months["march"] = 31; *months["april"] = 30; *months["may"] = 31; *months["june"] = 30; *months["july"] = 31; *months["august"] = 31; *months["september"] = 30; *months["october"] = 31; *months["november"] = 30; *months["december"] = 31; months->write_unlock(); months->read_lock(); cout << "june -> " << months["june"] << endl; shared_map<const char*, int, ltstr>::iterator cur = months.find("june"); shared_map<const char*, int, ltstr>::iterator prev = cur; shared_map<const char*, int, ltstr>::iterator next = cur; ++next; --prev; cout << "Previous (in alphabetical order) is " << (*prev).first << endl; cout << "Next (in alphabetical order) is " << (*next).first << endl; months->read_unlock(); }
Definition
62
Defined in the header shared_map.h.
Template parameters
Parameter Description Default Key The map's key type and value type. This is also defined as shared _map::key_type Data The map's data type. This is also defined as shared_map::data_type. Compare The key comparison function, a Strict Weak Ordering whose argument type is
key_type; it returns true if its first argument is less than its second argument, and false otherwise. This is also defined as shared_map::key_compare
less<Key>
Type requirements
Same requirements than for map.
Public base classes
sync
Members
All public members of the map class and of the sync class are exposed.
63
A.7 shared_hash_set <Key, Compare>
Description
A shared_hash_set is a shared memory implementation of a STL hash_set container. A shared_hash_set offers the same functionalities and properties than the hash_set. Please refer to the STL documentation for information on the hash_set class. shared_hash_set also offers synchronization methods that should be used before accessing the container if other processes are susceptible to access it concurrently. shared_hash_set objects are created, attached to and released from by calling the static methods of the SharedHash_setFactory class.
Definition
Defined in the header shared_hash_set.h.
Template parameters
Parameter Description Default Key The set's key type and value type. This is also defined as shared
_hash_set::key_type and shared_hash_set::value_type
Compare The key comparison function, a Strict Weak Ordering whose argument type is key_type; it returns true if its first argument is less than its second argument, and false otherwise. This is also defined as shared_hash_set::key_compare and shared_hash_set::value_compare.
less<Key>
Type requirements
Same requirements than for hash_set.
Public base classes
sync
Members
All public members of the hash_set class and of the sync class are exposed.
64
A.8 SharedVectorFactory<T >
Description
SharedVectorFactory is a static class used to create, attach to or detach from a shared_vector. Shared vectors are named entities to be identifiable by other processes.
Example
Process 1: shared_vector<int> *V = SharedVectorFactory<int>::create(“name”); Process 2: shared_vector<int> *V = SharedVectorFactory<int>::attach(“name”); ... SharedVectorFactory<int>::detach(“name”); Process 1: SharedVectorFactory<int>::detach(“name”);
Definition
Defined in the header shared_vector.h.
Template parameters
Parameter Description T The shared vector's value type: the type of object that is stored in the vector.
Type requirements
Same requirements than for vector.
Members
Member Description
shared_vector* attach(char* name) Attach to an existing shared_vector named name
shared_vector* create(char* name) Create new shared_vector named name
shared_vector* create(char* name, shared_vector&)
Copy constructor
shared_vector* create(char* name, size_t n)
Create new shared_vector named name with n elements
shared_vector* create(char* name, size_t n, const T& t)
Create new shared_vector named name with n copies of t
void detach(char* name) Detach from shared_vector named name
65
A.9 SharedStackFactory<T , Sequence>
Description
SharedStackFactory is a static class used to create, attach to or detach from a shared_stack. Shared stacks are named entities to be identifiable by other processes.
Example
Process 1: shared_stack<int> *S = SharedStackFactory<int>::create(“name”); Process 2: shared_stack<int> *S = SharedStackFactory<int>::attach(“name”); ... SharedStackFactory<int>::detach(“name”); Process 1: SharedStackFactory<int>::detach(“name”);
Definition
Defined in the header shared_stack.h.
Template parameters
Parameter Description Default T The shared stack's value type: the type of object that is stored
in the stack.
Sequence The type of the underlying container used to implement the stack.
shared_deque<T>
Type requirements
Same requirements than for stack.
Members
Member Description
shared_stack* attach(char* name) Attach to an existing shared_stack named name
shared_stack* create(char* name) Create new shared_stack named name
shared_stack* create(char* name, shared_stack&)
Copy constructor
shared_stack* create(char* name, Create new shared_stack named
66
size_t n) name with n elements shared_stack* create(char* name, size_t n, const T& t)
Create new shared_stack named name with n copies of t
void detach(char* name) Detach from shared_stack named name
67
A.10 SharedMapFactory map<Key, Data, Compare>
Description
SharedMapFactory is a static class used to create, attach to or detach from a shared_map. Shared maps are named entities to be identifiable by other processes.
Example
Process 1: shared_map<std::string, int> *S = SharedMapFactory<std::string, int>::create(“name”); Process 2: shared_map<std::string, int> *S = SharedMapFactory<std::string, int>::attach(“name”); ... SharedMapFactory<std::string, int>::detach(“name”); Process 1: SharedMapFactory<std::string, int>::detach(“name”);
Definition
Defined in the header shared_map.h.
Template parameters
Parameter Description Default Key The map's key type and value type. This is also defined as shared
_map::key_type and shared_map::value_type
Data The map's data type. This is also defined as shared_map::data_type.
Compare The key comparison function, a Strict Weak Ordering whose argument type is key_type; it returns true if its first argument is less than its second argument, and false otherwise. This is also defined as shared_map::key_compare and shared_map::value_compare.
less<Key>
Type requirements
Same requirements than for map.
Members
Member Description
shared_map* attach(char* name) Attach to an existing shared_map named name
shared_map* create(char* name) Create new shared_map named name
68
shared_map* create(char* name, const key_compare& comp)
Creates an empty shared_map named name, using comp as the key_compare object.
template <class InputIterator> shared_map* create(char* name, InputIterator f, InputIterator l)
Creates a shared_map named name with a copy of a range.
template <class InputIterator> shared_map* create(char* name, InputIterator f, InputIterator l, const key_compare& comp)
Creates a shared_map named name with a copy of a range, using comp as the key_compare object.
shared_map* create(char* name, shared_map&)
Copy constructor
void detach(char* name) Detach from shared_map named name
69
A.11 SharedHashSetFactory set<Key, Compare>
Description
SharedHashSetFactory is a static class used to create, attach to or detach from a shared_hash_set. Shared hash sets are named entities to be identifiable by other processes.
Example
Process 1: shared_hash_set<int> *S = SharedHashSetFactory<int>::create(“name”); Process 2: shared_hash_set<int> *S = SharedHashSetFactory<int>::attach(“name”); ... SharedHashSetFactory<int>::detach(“name”); Process 1: SharedHashSetFactory<int>::detach(“name”);
Definition
Defined in the header shared_hash_set.h.
Template parameters
Parameter Description Default Key The set's key type and value type. This is also defined as shared
_hash_set::key_type and shared_hash_set::value_type
Compare The key comparison function, a Strict Weak Ordering whose argument type is key_type; it returns true if its first argument is less than its second argument, and false otherwise. This is also defined as shared_hash_set::key_compare and shared_hash_set::value_compare.
less<Key>
Type requirements
Same requirements than for hash_set.
Members
Member Description shared_hash_set* attach(char* name)
Attach to an existing shared_hash_set named name
shared_hash_set* create(char* name)
Create new shared_hash_set named name
shared_hash_set* create(char* name, const key_compare& comp)
Creates an empty shared_hash_set named name, using comp as the
70
key_compare object. shared_hash_set* create(char* name, size_t n)
Create new shared_ash_set named name with at least n buckets
shared_hash_set* create(char* name, size_t n, const key_compare& comp)
Creates an empty shared_hash_set named name, with at least n buckets, using comp as the key_compare object.
template <class InputIterator> shared_hash_set* create(char* name, InputIterator f, InputIterator l)
Creates a shared_hash_set named name with a copy of a range.
template <class InputIterator> shared_hash_set* create(char* name, InputIterator f, InputIterator l, size_t n)
Creates a shared_hash_set named name with a copy of a range and a bucket count of at least n.
template <class InputIterator> shared_hash_set* create(char* name, InputIterator f, InputIterator l, size_t n, const key_compare& comp)
Creates a shared_hash_set named name with a copy of a range and a bucket count of at least n, using comp as the key_compare object.
shared_hash_set* create(char* name, shared_hash_set&)
Copy constructor
void detach(char* name) Detach from shared_hash_set named name
71
Appendix B. SHARED CONTAINERS LIBRARY SOURCE CODE
(EXCERPTS)
The latest version of the source code for the shared container library is available online at: http://bergeon.francois.perso.neuf.fr/containers/src.zip
B.1 safe_counter.h /* * safe_counter.h - Thread-safe counters * * --- (c) Francois Bergeon 2008 --- * ---- University of Liverpool ---- * * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #pragma once #include <windows.h> /* * Wrapper class around an interlocked counter * - Class is platform dependent. */ class safe_counter { private: // Counter is volatile and needs to be aligned // on a 32bit boundary for interlocked operations _declspec(align(32)) volatile LONG _counter; public: // Constructor - no need for interlocked access at this time inline safe_counter(long n = 0) : _counter(n) {}; // Assignment operator inline safe_counter& operator=(long n)
{ InterlockedExchange(&_counter, n); return *this; }; // Cast operator to const long inline operator const long()
{ return _counter; }; // Increment and decrement operators inline long operator++()
{ return InterlockedIncrement(&_counter); }; inline long operator--()
{ return InterlockedDecrement(&_counter); }; };
Listing 4: safe_counter.h
72
B.2 sync.h /* * sync.h - Simple read/write locking mechanism * * --- (c) Francois Bergeon 2008 --- * ---- University of Liverpool ---- * * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #pragma once #include "safe_counter.h" /* * sync class - implements a synchronization for shared containers * - Class is platform dependent. */ class sync { private: // Internal read/write sempahores safe_counter _readlock; safe_counter _writelock; long _lockingthreadid; long _lockingprocessid; // Standard loop delay is 10ms static const unsigned DELAY = 10;
73
public: // Obtain non-exclusive read lock with timeout // - More than one read lock is allowed at any single time // timeout - time out in milliseconds, -1 for infinite (default), 0 for no wait // bool read_lock(long timeout = -1); // Release non-exclusive read lock void read_unlock(); // Obtain exclusive write lock with timeout // - No read locks and only one write lock
// is allowed at any single time // timeout - time out in milliseconds, -1 for infinite (default), 0 for no wait // inline bool write_lock(long timeout = -1) { return write_lock(timeout, 0); };
// Check if the current thread owns the write lock bool is_locking_thread(); // Release exclusive write lock void write_unlock(); // Upgrade from a non-exclusive read lock to an
// exclusive write lock // - Thread must already own a read lock // - If another thread is already waiting for a wite lock it will be preempted // timeout - time out in milliseconds, 0 for no wait (default) // bool upgrade_lock(long timeout = 0); // Downgrade from an exclusive write lock to a non-exclusive read lock // - Thread must already own a write lock // - Read lock is guaranteed // void downgrade_lock(); private:
74
// Internal write lock with acceptable number of existing locks bool write_lock(long timeout, long existing); // Get thread and process id inline long get_threadid() { return GetCurrentThreadId(); } inline long get_processid() { return GetCurrentProcessId(); }
};
Listing 5: sync.h
B.3 sync.cpp /* * sync.cpp - Simple read/write locking mechanism * * --- (c) Francois Bergeon 2008 --- * ---- University of Liverpool ---- * * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #include "sync.h" // Obtain non-exclusive read lock with timeout // - More than one read lock is allowed at any single time // timeout - time out in milliseconds, 0 for infinite, 1 for no delay //
75
bool sync::read_lock(long timeout) { // Obtain maximum number of iterations to run long n = (timeout / DELAY) + 1; for (;;) { // Check that no write lock has been requested if (_writelock == 0) { // Obtain read lock ++_readlock; // Has no write lock been requested in the meantime? if (_writelock == 0) return true; // Release read lock and try again --_readlock; } // Timeout expired? if (timeout >= 0 && --n == 0) return false; // Wait Sleep(DELAY); } }; // Release non-exclusive read lock void sync::read_unlock() { // Safety if (--_readlock == -1) ++_readlock; }
76
// Obtain exclusive write lock with timeout // - No read locks and only one write lock is allowed at any single time // timeout - time out in milliseconds, 0 for infinite, 1 for no delay // existing - acceptable number of existing locks (0:normal, 1:upgrade) // bool sync::write_lock(long timeout, long existing) { // Has this thread already obtained an exclusive write lock? if (is_locking_thread()) { // Increment lock counter so that the next call to // write_unlock() does not release the lock ++_writelock; return true; } // Obtain maximum number of iterations to run long n = (timeout / DELAY) + 1; for (;;) { // Try to obtain an exclusive write lock // or preempt other threads if we are upgrading // from a non-exclusive one while (++_writelock > 1 && existing == 0) { --_writelock; // Timeout expired? if (timeout >= 0 && --n == 0) return false; // Wait Sleep(DELAY); } // Then wait for all readers to release their non-exlusive read locks while (_readlock > existing)
77
{ // Timeout expired? if (timeout >= 0 && --n == 0) { // Release write lock --_writelock; return false; } // Wait Sleep(DELAY); } // Write lock obtained, store locking thread id _lockingthreadid = get_threadid(); _lockingprocessid = get_processid(); return true; } }; // Check if the current thread owns the write lock bool sync::is_locking_thread() { return (_writelock > 0 && _lockingthreadid == get_threadid() &&
_lockingprocessid == get_processid()); } // Release exclusive write lock void sync::write_unlock() { long n = --_writelock; // Safety if (n == -1) ++_writelock; // Reset thread and process id if (n == 0) _lockingthreadid = _lockingprocessid = -1; }
78
// Upgrade from non-exclusive read lock to exclusive write lock // - Thread must already own a read lock // - If another thread is already waiting for a wite lock it will be preempted // - An infinite timeout may lead to a running condition // timeout - time out in milliseconds, 0 for no wait // bool sync::upgrade_lock(long timeout) { // Obtain a write lock with one existing read lock (ours) if (!write_lock(timeout, 1)) return false; // Release our read lock read_unlock(); // Lock upgraded return true; } // Downgrade from an exclusive write lock to a non-exclusive read lock // - Thread must already own a write lock // - Read lock is guaranteed // void sync::downgrade_lock() { ++_readlock; write_unlock(); }
Listing 6: sync.cpp
79
B.4 heap_parameters.h /* * heap_parameters.h - Parameters for shared heap * * --- (c) Francois Bergeon 2008 --- * ---- University of Liverpool ---- * * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #include <stddef.h> // These variables can be modified to suit specific needs char* _heapname = "SHARED_CONTAINERS"; // Arbitrary heap name size_t _heapsize = 67108864; // 64Mb size_t _chunksize = 64; // 64 bytes void* _baseaddress = (void*)0x01100000; // Arbitrary high for debug
Listing 7: heap_parameters.h
B.5 shared_heap.h /* * shared_heap.h - Heap implementation in shared memory * * --- (c) Francois Bergeon 2008 --- * ---- University of Liverpool ----
80
* * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #pragma once #ifdef WIN32 #include <windows.h> #endif /* * shared_heap class - implements a heap in shared memory * * This class is platform dependent. * */ class shared_heap { private: class shared_header; // Forward class declaration // Class members shared_header* _header; // Pointer to shared header void* _heap; // Address of shared heap long _lasterror; // Last error caught #ifdef WIN32 HANDLE _mapping; // Handle of Win32 mapping object #endif // Standard loop delay is 20ms static const unsigned DELAY = 20;
81
public: // Public methods shared_heap(const char* heapname, size_t heapsize = 0x100000,
size_t chunksize = 0x1000, void* baseaddress = NULL); ~shared_heap(void); void* allocate(size_t size); void deallocate(void* head, size_t size); void get_params(long** longparam, void*** ptrparam); #ifdef _DEBUG void dump(); #endif private: // Private methods void cleanup(); void lock(); void unlock(); };
Listing 8: shared_heap.h
B.6 shared_heap.cpp /* * shared_heap.cpp - Heap implementation in shared memory * * --- (c) Francois Bergeon 2008 --- * ---- University of Liverpool ---- * * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a
82
* Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * * This class implements a heap of segregated storage in shared memory. * It uses WIN32 primitives and is platform dependent. * */ #include "shared_heap.h" #include "safe_counter.h" #ifndef BYTE #define BYTE unsigned char #endif // The signature is used to confirm that an existing memory // mapping has been created by this same code #define SHAREDHEAP_SIGNATURE "**FBSH**" // Indicator of last chunk in chunk list #define LAST_CHUNK ((LPVOID)~0) // Macro to roundup value #define ROUNDUP(a,b) ((a)%(b)==0?(a):((((a)/(b))+1)*(b))) // // The shared header is located at the beginning of the shared memory mapped segment // All processes accessing shared containers rely on the shared header to map // shared memory to the same virtualized memory space and to access existing containers // __declspec(align(32)) volatile class shared_heap::shared_header { public: char _signature[sizeof(SHAREDHEAP_SIGNATURE)]; // Signature that identifies view as a shared heap void* _baseaddress; // Common base address to be used by all processes void* _head; // Pointer to first available chunk size_t _heapsize, _chunksize; // Size of heap and chunk size
83
long _longparam; // Client long parameter void* _ptrparam; // Client pointer parameter safe_counter _numprocesses; // Number of processes currently accessing shared memory safe_counter _allocating; // 1 if allocation in progress, 0 otherwise char _filename[MAX_PATH]; // Temp filename }; // // Shared heap constructor // // heapname - name of shared heap // heapsize - size of shared heap in bytes (defaults to 0x100000 = 1Mb) // chnksize - size of segregated storage chunks (defauts to 0x1000 = 4Kb) // baseaddress - address of shared mempory mapping of NULL if default // shared_heap::shared_heap(const char* heapname, size_t heapsize, size_t chunksize, void* baseaddress) { // Flag for creation of new shared heap BOOL bCreateNew = FALSE; // Size of shared header is rounded up to proper allocation granularity SYSTEM_INFO SystemInfo; GetSystemInfo(&SystemInfo); DWORD dwHeaderSize = ROUNDUP(sizeof(shared_header), SystemInfo.dwAllocationGranularity); // Buffer for temp filename and wide char heap name TCHAR szTmpFile[MAX_PATH]; TCHAR szHeapName[MAX_PATH]; // Convert heap name to wide char for Win32 API calls MultiByteToWideChar(CP_ACP, 0, heapname, -1, szHeapName, MAX_PATH); // Initialize members _lasterror = 0; _mapping = NULL; _header = NULL; _heap = NULL;
84
try { // Open file mapping _mapping = OpenFileMapping(FILE_MAP_WRITE, // RW access FALSE, // Handle cannot be inherited szHeapName); // Name of file mapping object // Obtain last system error _lasterror = GetLastError(); // Unable to open file mapping, attempt to create temp file if (_mapping == NULL && _lasterror == ERROR_FILE_NOT_FOUND) { // Create file mapping bCreateNew = TRUE; // Check requested heap size if (heapsize == NULL) { SetLastError(ERROR_BAD_LENGTH); throw; } // Create temporary file for mapping TCHAR szTmpPath[MAX_PATH-14]; // Get temp dir & file GetTempPath(MAX_PATH-14, szTmpPath); GetTempFileName(szTmpPath, szHeapName, 0, szTmpFile); // Create temp file HANDLE hFile = CreateFile(szTmpFile, // Temp file name GENERIC_WRITE | GENERIC_READ, // RW Access FILE_SHARE_WRITE, // Share mode NULL, // Optional security attributes CREATE_ALWAYS, // Create file FILE_ATTRIBUTE_TEMPORARY, // Temporary file NULL); // Flags & attributes // Obtain last system error _lasterror = GetLastError();
85
// Exit if we could not create temp file if (hFile == INVALID_HANDLE_VALUE) throw; // Add header size to desired size __int64 qwMaxSize = heapsize + dwHeaderSize; DWORD dwSizeHigh = (qwMaxSize & 0xFFFFFFFF00000000) >> 32; // Hi dword DWORD dwSizeLow = (qwMaxSize & 0x00000000FFFFFFFF); // Low dword // Create new file mapping object _mapping = CreateFileMapping(hFile, // Handle to temp file NULL, // Optional security attributes PAGE_READWRITE, // Page protection dwSizeHigh, // Hi-order DWORD of size dwSizeLow, // Low-order DWORD of size szHeapName); // Name of file mapping object // Obtain last system error _lasterror = GetLastError(); // Close underlying temp file CloseHandle(hFile); } // Exit function if OpenFileMapping or CreateFileMapping failed if (_mapping == NULL) throw; // Map view of file to obtain/create header _header = (shared_header *)MapViewOfFileEx(_mapping, // File mapping object FILE_MAP_WRITE, // RW access 0, 0, // Offset 0 dwHeaderSize, // Header size NULL); // Any base address // Unable to map view of file: cleanup & exit if (_header == NULL) { _lasterror = GetLastError(); throw;
86
} if (bCreateNew) { // Map view of file to target base address _heap = MapViewOfFileEx(_mapping, // File mapping object FILE_MAP_WRITE, // RW access 0, dwHeaderSize, // Offset (must be aligned to allocation granularity) 0, // Full size baseaddress); // Target base address or NULL // Save last error if failed _lasterror = GetLastError(); if (_heap == NULL) throw; // Populate header _header->_baseaddress = _heap; // Now common base address _header->_head = LAST_CHUNK; // Storage not initialized _header->_heapsize = heapsize; // Heap size _header->_chunksize = max(chunksize, sizeof(void**)); // Chunk size _header->_longparam = 0; // Set client DWORD parameter _header->_ptrparam = NULL; // Set client LPVOID parameter _header->_numprocesses = 1; // Currently 1 process _header->_allocating = 0; // No allocation currently in progress WideCharToMultiByte(CP_ACP, 0, szTmpFile, -1, _header->_filename, MAX_PATH, NULL, NULL); // Convert & store temp filename // Initialize segregated storage deallocate(_heap, heapsize); // Insert signature last to prevent race conditions with other processes strcpy_s(_header->_signature, sizeof(SHAREDHEAP_SIGNATURE), SHAREDHEAP_SIGNATURE); // Signature } else { // Check signature or fail if (strcmp(_header->_signature, SHAREDHEAP_SIGNATURE) != 0) { _lasterror = ERROR_BAD_FORMAT; throw;
87
} // Increment number of client processes if (++_header->_numprocesses == 1) { // A process is currently closing the shared memory _lasterror = ERROR_FILE_NOT_FOUND; throw; } // Map view of file to target base address _heap = MapViewOfFileEx(_mapping, // File mapping object FILE_MAP_WRITE, // RW access 0, dwHeaderSize, // Offset (must be aligned to allocation granularity) 0, // Full size _header->_baseaddress); // Common base address // Save last error if failed _lasterror = GetLastError(); if (_heap == NULL) throw; } } catch (...) { // Cleanup cleanup(); // Re-set error code SetLastError(_lasterror); } } // Standard destructor shared_heap::~shared_heap(void) { cleanup(); } // // Cleanup is called by destructor or
88
// if an exception is caught in the constructor // void shared_heap::cleanup() { // Buffer for temp filename TCHAR szTmpFile[MAX_PATH]; // Initialize as an empty string szTmpFile[0] = '\0'; // Unmap heap if (_heap != NULL) UnmapViewOfFile(_heap); // Unmap header if (_header != NULL) { // Are we the last process to use this shared memory file? if (--_header->_numprocesses == 0) { // If so, convert temp filename so we can delete it MultiByteToWideChar(CP_ACP, 0, _header->_filename, -1, szTmpFile, MAX_PATH); } // Unmap header UnmapViewOfFile((LPVOID)_header); } // Close file mapping if (_mapping != NULL) CloseHandle(_mapping); // Reset member variables _heap = NULL; _header = NULL; _mapping = NULL; // Delete temp file if we are the last user process
89
if (szTmpFile[0] != '\0') DeleteFile(szTmpFile); } // // Allocate memory // // size - number of bytes to allocate // void* shared_heap::allocate(size_t size) { // Synchronization lock(); // Out of memory? if (_header->_head == LAST_CHUNK) { unlock(); return NULL; } // Roundup allocation size to a multiple of chunk size size = ROUNDUP(size, _header->_chunksize); // Special case: allocation of a single chunk if (size == _header->_chunksize) { // Obtain head chunk void** p = (void**)_header->_head; // Move head to point to next chunk _header->_head = *p; // Return former head chunk unlock(); return (void*)p; } // Number of contiguous chunks to allocate if > 1
90
unsigned chunks = size / _header->_chunksize; // Start at head chunk and look for contiguous block of chunks void* head = _header->_head; // Start of contiguous block void** previous = &_header->_head; // Pointer to previous chunk // Loop do { // Loop variables void** p; // Index pointer to chunks unsigned n; // Counter for contiguous chunks // Loop through contiguous chunks for (p = (void**)head, n = 1; *p == ((BYTE*)p + _header->_chunksize); p = (void**)*p, ++n) { // Return if we found enough contiguous chunks if (n == chunks) { // Set chunk before block to point to next chunk *previous = *p; // Return start of chunk block unlock(); return head; } } // Continue searching from next chunk forward previous = p; head = *p; // Stop when we reach the end of the heap } while (head != LAST_CHUNK); // No contiguous chunks found, allocation failed unlock(); return NULL; } // // Deallocate buffer - can also be used to initialize segregated
91
// storage for the entire heap // // head - pointer to buffer // size - buffer size // void shared_heap::deallocate(void* head, size_t size) { // Synchronization lock(); // Roundup size to an integer number of chunks size = ROUNDUP(size, _header->_chunksize); // Calculate address of last chunk to be deallocated void** toe = (void**)((BYTE*)head + size - _header->_chunksize); // Loop variables void** previous = NULL; // Address of previous chunk void* p; // Index pointer to chunks // Go through all chunks from head to toe for (p = head; p <= toe; p = ((BYTE*)p + _header->_chunksize)) { // Update previous chunk so it points to current chunk if (previous != NULL) *previous = p; // Store address of new previous chunk previous = (void**)p; } // Make last chunk point to previous head *toe = _header->_head; // Make head point to first chunk _header->_head = head; unlock(); } //
92
// Obtain client parameters from shared header // // longparam - pointer to long parameter // ptrparam - pointer to pointer parameter // void shared_heap::get_params(long** longparam, void*** ptrparam) { *longparam = &_header->_longparam; *ptrparam = &_header->_ptrparam; } // Lock shared heap. Wait if busy. No timeout void shared_heap::lock() { if (_header != NULL) { while (++_header->_allocating > 1) { if (--_header->_allocating == 0) continue; Sleep(DELAY); } } } // Unlock shared heap void shared_heap::unlock() { if (_header != NULL) --_header->_allocating; }
Listing 9: shared_heap.cpp
93
B.7 shared_pool.h /* * shared_pool.h - Object allocation in shared memory * * --- (c) Francois Bergeon 2008 --- * ---- University of Liverpool ---- * * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #pragma once #include <map> #include <string> #include "shared_heap.h" #include "shared_allocator.h" #include "sync.h" /* * shared_map class * - derives from the map and sync classes * - implements the map class with the shared_allocator template parameter */ template<class Key, class Data, class Compare=std::less<Key>> class shared_map : public sync, public std::map<Key, Data, Compare, shared_allocator<std::pair<const Key, Data>>> {}; /* * Shared pool class - Provides singleton allocation interface * to shared heap and manages map of shared objects * */ class shared_pool
94
{ private: // shared_object class describes each shared objects class shared_object; // Map of objects stored in shared memory typedef shared_map<std::string, shared_object*> container_map; // Shared heap and map of shared objects static shared_heap* _heap; static container_map* _containermap; public: // Allocator interface to shared heap static void* allocate(size_t size); static void deallocate(void* buffer, size_t size); static size_t max_size(); // Insert, retrieve and release objects in shared map static bool insert(char *name, void* object, size_t size); static void* retrieve(char *name, size_t size); static void* release(char *name); private: // Lazy initialization static void init(); };
Listing 10: shared_pool.h
B.8 shared_pool.cpp /* * shared_pool.cpp - Object allocation in shared memory *
95
* --- (c) Francois Bergeon 2008 --- * ---- University of Liverpool ---- * * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ // Force Microsoft VC++ 6 to turn off iterator checking #ifdef WIN32 #define _HAS_ITERATOR_DEBUGGING 0 #define _SECURE_SCL 0 #endif #include "shared_pool.h" // External parameters used to construct a shared heap // These variables can be modified to suit specific needs extern char* _heapname; // Arbitrary heap name extern size_t _heapsize; // 10Mb extern size_t _chunksize; // 32 bytes extern void* _baseaddress; // Arbitrary high for debug // Class for shared object stored in container map class shared_pool::shared_object { public: void* _object; // Pointer to object size_t _size; // Size of object safe_counter _refcount; // Usage count shared_object(void* object, size_t size): _object(object), _size(size), _refcount(1) {};
96
shared_object() {}; ~shared_object() {}; }; // Initialize static members shared_heap* shared_pool::_heap = NULL; shared_pool::container_map* shared_pool::_containermap = NULL; // // Allocate buffer from shared heap // // size - number of bytes to allocate // void* shared_pool::allocate(size_t size) { // Perform lazy intialization if shared heap is not initialized if (_heap == NULL) init(); // Allocate memory and return return _heap->allocate(size); } // // Deallocate buffer // // head - pointer to buffer // size - buffer size // void shared_pool::deallocate(void* buffer, size_t size) { // Shared heap must be initialized if (_heap == NULL) return; // Deallocate buffer _heap->deallocate(buffer, size);
97
}; // // Return heap size // size_t shared_pool::max_size() { return _heapsize; } // // Insert shared object in container map // // name - name of object // object - pointer to object // size - size of object // bool shared_pool::insert(char *name, void* object, size_t size) { // Perform lazy intialization if shared heap is not initialized if (_heap == NULL) init(); // Look for entry - obtain non-exclusive lock first // Return failure if a container with the same name is already present in the shared container map std::string s(name); _containermap->read_lock(); if (_containermap->find(s) != _containermap->end()) { // Release read lock and return failure _containermap->read_unlock(); return false; } // Allocate and build new shared object void* p = allocate(sizeof(shared_object)); if (p == NULL)
98
{ // Release read lock and return failure _containermap->read_unlock(); return false; } shared_object* so = new(p) shared_object(object, size); // Upgrade to an exclusive lock if (!_containermap->upgrade_lock()) { // If upgrade failed we have to do it manually // First release the read lock - this may give another process a write lock _containermap->read_unlock(); // Then obtain a write lock the old fashioned way _containermap->write_lock(); // Check for name again in case it was inserted concurrently if (_containermap->find(s) != _containermap->end()) { // Release write lock and deallocate object, return failure _containermap->write_unlock(); deallocate(p, sizeof(shared_object)); return false; } } // Insert shared object and release lock (*_containermap)[s] = so; _containermap->write_unlock(); // Return success return true; } // // Retrieve shared object from container map //
99
// name - name of object // size - size of object // void* shared_pool::retrieve(char *name, size_t size) { // Perform lazy intialization if shared heap is not initialized if (_heap == NULL) init(); // Look for entry - obtain non-exclusive lock _containermap->read_lock(); std::string s(name); container_map::iterator it = _containermap->find(s); // Return NULL if no container with that name is present in the shared container map if (it == _containermap->end()) { // Release read lock and return failure _containermap->read_unlock(); return NULL; } // Obtain pointer to shared object and release read lock shared_object* so = it->second; _containermap->read_unlock(); // Check shared object and return NULL if size does not match if (so == NULL || so->_size != size) return NULL; // Increment refcount and return pointer to object ++so->_refcount; return so->_object; } // // Release object by decrementing its refcount. If refcount reaches zero,
100
// return pointer to object so its destructor can be called. // // name - name of object // void* shared_pool::release(char *name) { // Shared heap must be initialized if (_heap == NULL) return NULL; // Look for entry - obtain non-exclusive lock first std::string s(name); _containermap->read_lock(); container_map::iterator it = _containermap->find(s); // Return NULL if no container with that name is present in the shared container map if (it == _containermap->end()) { // Release read lock and return failure _containermap->read_unlock(); return NULL; } // Obtain pointer to shared object and decrement refcount shared_object* so = it->second; if (so == NULL || --so->_refcount > 0) { // Release read lock and return NULL if object still in use _containermap->read_unlock(); return NULL; } // // refcount == 0 - discard object // // Retain pointer to object
101
void* p = so->_object; // Upgrade to an exclusive lock if (!_containermap->upgrade_lock()) { // If upgrade failed we have to do it manually // First release the read lock - this may give another process a write lock _containermap->read_unlock(); // Then obtain a write lock the old fashioned way _containermap->write_lock(); // Get shared object again in case it was removed or attached concurrently if (_containermap->find(s) == _containermap->end() || so->_refcount > 0) { // Return NULL if object not found or still in use _containermap->write_unlock(); return NULL; } } // Erase map entry and release lock _containermap->erase(it); _containermap->write_unlock(); // Deallocate object deallocate(so, sizeof(shared_object)); // Return pointer to object for destruction return p; } // // Lazy initialization of shared heap // void shared_pool::init() { // Pointers to shared client parameters
102
long* longparam; void** ptrparam; // Build new shared heap from external parameters _heap = new shared_heap(_heapname, _heapsize, _chunksize, _baseaddress); // Obtain pointers to client parameters in shared header _heap->get_params(&longparam, &ptrparam); // Create container map if not initialized if (*ptrparam == NULL) { _containermap = new(_heap->allocate(sizeof(container_map))) container_map; // Store pointer to map in pParam *ptrparam = _containermap; } else { // Or attach to existing container map _containermap = (container_map*)*ptrparam; } }
Listing 11: shared_pool.cpp
B.9 shared_allocator.h /* * shared_allocator.h - STL allocator in shared memory * * --- (c) Francois Bergeon 2008 --- * ---- University of Liverpool ---- * * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY
103
* * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #pragma once #include "shared_pool.h" class shared_pool; template<class T> class shared_allocator { public: typedef T value_type; typedef unsigned int size_type; typedef ptrdiff_t difference_type; typedef T* pointer; typedef const T* const_pointer; typedef T& reference; typedef const T& const_reference; pointer address(reference r) const { return &r; } const_pointer address(const_reference r) const {return &r;} shared_allocator() throw() {}; template<class U> shared_allocator(const shared_allocator<U>& t) throw() {}; ~shared_allocator() throw() {}; // space for n Ts pointer allocate(size_t n, const void* hint=0) { return(static_cast<pointer> (shared_pool::allocate(n*sizeof(T)))); }
104
// deallocate n Ts, don't destroy void deallocate(pointer p, size_type n) { shared_pool::deallocate((LPVOID)p, n*sizeof(T)); return; } // initialize *p by val void construct(pointer p, const T& val) { new(p) T(val); } // destroy *p but don't deallocate void destroy(pointer p) { p->~T(); } size_type max_size() const throw() { return shared_pool::max_size(); } template<class U> struct rebind { typedef shared_allocator<U> other; }; }; template<class T> bool operator ==(const shared_allocator<T>& a, const shared_allocator<T>& b) throw() { return(TRUE); } template<class T> bool operator !=(const shared_allocator<T>& a, const shared_allocator<T>& b) throw() { return(FALSE); }
Listing 12: shared_allocator.h
105
B.10 container_factories.h /* * containers_factories.h - STL containers in shared memory * * --- (c) Francois Bergeon 2008 --- * ---- University of Liverpool ---- * * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #pragma once #include "shared_pool.h" // Placement new in the shared pool #define SHARED_NEW new(shared_pool::allocate(sizeof(container_type))) /* * SharedContainerFactory - parameterized abstract factory class * - Used to create new shared containers or attach to existing ones. * - Methods of this class are static and call static methods of shared_pool. * - Class is meant to be derived into specific shared container factory classes. */ template<class C> class _SharedContainerFactory { public: typedef C container_type; // Container's type - this type is available to all derived classes // Create new shared container and assign specified name // name - name of shared container // static container_type* create(char* name)
106
{ // Allocate new container in shared memory container_type* c = SHARED_NEW container_type(); // Add container to shared pool and return return insert_container(name, c); }; // Create new shared container with copy constructor // name - name of shared container // cont - container to copy from // static container_type* create(char* name, const container_type& cont) { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type(cont); // Add container to shared pool and return return insert_container(name, c); }; // Attach to existing shared container by name // name - name of shared container // static container_type *attach(char* name) { // Retrieve container from shared pool return (container_type*)shared_pool::retrieve(name, sizeof(container_type)); }; // Detach from shared container - release if last user // name - name of shared container // static void detach(char* name) { // Detach from container in sahred pool container_type* c = (container_type*)shared_pool::release(name); // shared_pool returns pointer to the container if we are the last user, NULL otherwise if (c != NULL)
107
destroy_container(c); } protected: // Attempt to insert container in shared pool // name - name of shared container // c - pointer to shared container // static container_type* insert_container(char* name, container_type *c) { // Add container to shared pool - destroy it and return NULL if insertion failed if (!shared_pool::insert(name, c, sizeof(container_type))) { destroy_container(c); return NULL; } // Returned newly created container return c; } // Destroy container and deallocate // c - pointer to shared container // static void destroy_container(container_type* c) { // Call destructor on shared container c->~container_type(); // Deallocate memory used by shared container object shared_pool::deallocate(c, sizeof(container_type)); }; }; /* * SharedSequenceContainerFactory - parameterized abstract factory class * - Specialization of the SharedContainerFactory class for sequence containers. * - Implements the pre-allocation and replication constructors offered * by sequence containers.
108
*/ template<class C> class _SharedSequenceContainerFactory : public _SharedContainerFactory<C> { public: typedef typename container_type::value_type value_type; // Container's value type typedef typename container_type::size_type size_type; // Container's size_type // Redefinition of base class static create methods static container_type* create(char* name)
{ return _SharedContainerFactory<container_type>::create(name); }; static container_type* create(char* name, const container_type& c)
{ return _SharedContainerFactory<container_type>::create(name, c); }; // Create new shared container with pre-allocation // name - name of shared container // n - number of objects to pre-allocate // static container_type* create(char* name, size_type n) { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type(n); // Add container to shared pool and return return insert_container(name, c); }; // Create new shared container with replication // name - name of shared container // n - number of objects to replicate // t - object to replicate // static container_type* create(char* name, size_type n, value_type& t) { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type(n, t); // Add container to shared pool and return return insert_container(name, c);
109
}; }; /* * SharedAssociativeContainerFactory - parameterized abstract factory class * - Specialization of the SharedContainerFactory class for associative containers. * - Implements constructors with copy of range and specified key_compare function * offered by associative containers. */ template<class C> class _SharedAssociativeContainerFactory : public _SharedContainerFactory<C> { public: typedef typename container_type::key_compare key_compare; // Container's key_compare function // Redefinition of base class static create methods static container_type* create(char* name)
{ return _SharedContainerFactory<container_type>::create(name); }; static container_type* create(char* name, container_type& c)
{ return _SharedContainerFactory<container_type>::create(name, c); }; // Create new shared container with key_compare object // name - name of shared container // comp - key_compare object // static container_type* create(char* name, const key_compare& comp) { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type(comp); // Add container to shared pool and return return insert_container(name, c); } // Create new shared container with copy of range // name - name of shared container // f - first iterator // l - last iterator
110
// template<class InputIterator>
static container_type* create(char* name, InputIterator f, InputIterator l) { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type<InputIterator>(f, l); // Add container to shared pool and return return insert_container(name, c); }; // Create new shared container with copy of range and key_compare object // name - name of shared container // f - first iterator // l - last iterator // comp - key_compare object // template<class InputIterator>
static container_type* create(char* name, InputIterator f, InputIterator l, const key_compare& comp)
{ // Allocate new container in shared memory container_type* c = SHARED_NEW container_type<InputIterator>(f, l, comp); // Add container to shared pool and return return insert_container(name, c); }; }; /* * SharedHashContainerFactory - parameterized abstract factory class * - Specialization of the SharedContainerFactory class for hash-based associative containers. * - Implements constructors with copy of range and specified number of buckets, * hash and key_equal functions offered by hash-based associative containers. */ template<class C> class _SharedHashContainerFactory : public _SharedContainerFactory<C> {
111
public: typedef typename container_type::size_type size_type; // Container's size_type typedef typename container_type::key_compare key_compare; // Container's hash function // Redefinition of base class static create methods static container_type* create(char* name)
{ return _SharedContainerFactory<container_type>::create(name); }; static container_type* create(char* name, container_type& c)
{ return _SharedContainerFactory<container_type>::create(name, c); }; // Create new shared container with a number of buckets // name - name of shared container // n - number of buckets // static container_type* create(char* name, size_type n) { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type(n); // Add container to shared pool and return return insert_container(name, c); } // Create new shared container with a number of buckets and a hash function // name - name of shared container // n - number of buckets // comp - key comparison function // static container_type* create(char* name, size_type n, const key_compare& comp) { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type(n, comp); // Add container to shared pool and return return insert_container(name, c); } // Create new shared container with copy of range // name - name of shared container
112
// f - first iterator // l - last iterator // template<class InputIterator> static container_type* create(char* name, InputIterator f, InputIterator l) { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type<InputIterator>(f, l); // Add container to shared pool and return return insert_container(name, c); }; // Create new shared container with copy of range and number of buckets // name - name of shared container // f - first iterator // l - last iterator // n - number of buckets // template<class InputIterator>
static container_type* create(char* name, InputIterator f, InputIterator l, size_type n) { // Allocate new container in shared memory container_type* c = SHARED_NEW container_type<InputIterator>(f, l, n); // Add container to shared pool and return return insert_container(name, c); }; // Create new shared container with copy of range, number of buckets // and hash function // name - name of shared container // f - first iterator // l - last iterator // n - number of buckets // comp - key comparison function // template<class InputIterator>
static container_type* create(char* name, InputIterator f, InputIterator l, size_type n, const key_compare& comp)
113
{ // Allocate new container in shared memory container_type* c = SHARED_NEW container_type<InputIterator>(f, l, n, comp); // Add container to shared pool and return return insert_container(name, c); }; };
Listing 13: container_factories.h
B.11 shared_deque.h /* * shared_deque.h - STL containers in shared memory * * --- (c) Francois Bergeon 2008 --- * ---- University of Liverpool ---- * * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #pragma once #include <deque> #include "container_factories.h" /* * shared_deque class * - derives from the deque and sync classes
114
* - implements the deque class with the shared_allocator template parameter */ template<class T> class shared_deque : public sync, public std::deque<T, shared_allocator<T>> {}; /* * ShareddequeFactory - factory class to create shared deques * - Used to create new shared deques or attach to existing ones. * - Derives from the _SharedSequenceContainerFactory to implement * pre-allocation and replication constructors */ template<class T> class SharedDequeFactory : public _SharedSequenceContainerFactory<shared_deque<T>> {};
Listing 14: shared_deque.h
B.12 shared_stack.h /* * shared_stack.h - STL containers in shared memory * * --- (c) Francois Bergeon 2008 --- * ---- University of Liverpool ---- * * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */
115
#pragma once #include <stack> #include "shared_deque.h" /* * shared_stack class * - derives from the stack and sync classes * - default sequence class is shared_deque */ template<class T, class Sequence=shared_deque<T>> class shared_stack : public sync, public std::stack<T, Sequence> {}; /* * SharedStackFactory - factory class to create shared stacks * - Used to create new shared stacks or attach to existing ones. * - Derives from the _SharedSequenceContainerFactory to implement * pre-allocation and replication constructors * - Default sequence class is shared_deque */ template<class T, class Sequence=shared_deque<T>> class SharedStackFactory : public _SharedSequenceContainerFactory<shared_stack<T, Sequence>> {};
Listing 15: shared_stack.h
116
B.13 shared_set.h /* * shared_set.h - STL containers in shared memory * * --- (c) Francois Bergeon 2008 --- * ---- University of Liverpool ---- * * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #pragma once #include <set> #include "container_factories.h" /* * shared_set class * - derives from the set and sync classes * - implements the set class with the shared_allocator template parameter */ template<class Key, class Compare=std::less<Key>> class shared_set : public sync, public std::set<Key, Compare, shared_allocator<Key>> {}; /* * shared_multiset class * - derives from the multiset and sync classes * - implements the multiset class with the shared_allocator template parameter */
117
template<class Key, class Compare=std::less<Key>> class shared_multiset : public sync, public std::multiset<Key, Compare, shared_allocator<Key>> {}; /* * SharedSetFactory - factory class to create shared sets * - Used to create new shared sets or attach to existing ones. * - Derives from the _SharedAssociativeContainerFactory to implement * constructors with copy of range and specified key_compare function * offered by associative containers. */ template<class Key, class Compare=std::less<Key>> class SharedSetFactory : public _SharedAssociativeContainerFactory<shared_set<Key, Compare>> {}; /* * SharedMultisetFactory - factory class to create shared multisets * - Used to create new shared multisets or attach to existing ones. * - Derives from the _SharedAssociativeContainerFactory to implement * constructors with copy of range and specified key_compare function * offered by associative containers. */ template<class Key, class Compare=std::less<Key>> class SharedMultisetFactory : public _SharedAssociativeContainerFactory<shared_multiset<Key, Compare>> {};
Listing 16: shared_set.h
B.14 shared_hash_map.h /* * shared_hash_map.h - STL containers in shared memory * * --- (c) Francois Bergeon 2008 --- * ---- University of Liverpool ---- *
118
* IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * */ #pragma once #include <hash_map> #include "container_factories.h" /* * shared_hash_map class * - derives from the hash_map and sync classes * - implements the hash_map class with the shared_allocator template parameter */ template<class Key, class Data, class Compare=stdext::hash_compare<Key, std::less<Key>>> class shared_hash_map : public sync, public stdext::hash_map<Key, Data, Compare, shared_allocator<std::pair<const Key, Data>>> {}; /* * shared_hash_multimap class * - derives from the hash_multimap and sync classes * - implements the hash_multimap class with the shared_allocator template parameter */ template<class Key, class Data, class Compare=stdext::hash_compare<Key, std::less<Key>>> class shared_hash_multimap : public sync, public stdext::hash_multimap<Key, Data, Compare, shared_allocator<std::pair<const Key, Data>>> {}; /* * SharedHashMapFactory - factory class to create shared hash maps * - Used to create new shared hash maps or attach to existing ones. * - Derives from the _SharedHashContainerFactory to implement * constructors with copy of range and specified key_compare function * offered by associative containers.
119
*/ template<class Key, class Data, class Compare=stdext::hash_compare<Key, std::less<Key>>>
class SharedHashMapFactory : public _SharedHashContainerFactory<shared_hash_map<Key, Data, Compare>> {}; /* * SharedHashMultimapFactory - factory class to create shared hash multimaps * - Used to create new shared hash multimaps or attach to existing ones. * - Derives from the _SharedHashContainerFactory to implement * constructors with copy of range and specified key_compare function * offered by associative containers. */ template<class Key, class Data, class Compare=stdext::hash_compare<Key, std::less<Key>>>
class SharedHashMultimapFactory : public _SharedHashContainerFactory<shared_hash_multimap<Key, Data, Compare>> {};
Listing 17: shared_hash_map.h
120
Appendix C. PERFORMANCE EVALUATION RESULTS
Table 3 presents the raw performance evaluation results discussed above (averages are in bold).
Local memory Shared memory (create) Shared memory (attach) create populate verify erase create populate verify erase attach populate verify detach vector<int> 1,000,000 non-repetitive elements 1 0 17 1 0 0 17 1 0 0 14 1 0 2 0 12 1 0 0 21 1 0 0 18 1 0 3 0 12 1 0 0 18 1 0 0 15 1 0 4 0 12 1 0 0 22 1 0 0 21 1 0 5 0 12 3 0 0 29 1 0 0 4 1 0 6 0 12 1 0 0 5 1 0 0 31 1 0 7 0 14 1 0 0 5 1 0 0 5 1 0 8 0 12 1 0 0 40 1 0 0 40 1 0 9 0 12 1 0 0 5 1 0 0 4 1 0 10 0 12 1 0 0 4 1 0 0 4 1 0 11 0 13 1 0 0 5 1 0 0 5 1 0 12 0 12 1 0 0 57 1 0 0 60 1 0 13 0 13 1 0 0 4 1 0 0 5 1 0
Avg 0 13 1 0 0 18 1 0 0 17 1 0 map<string,int> 1,000,000 non-repetitive elements 1 0 1670 1091 394 0 1512 1107 187 0 1777 1306 0 2 0 1685 1091 394 0 1657 1261 194 0 1729 1307 0 3 0 1726 1091 394 0 1754 1301 198 0 1790 1317 0 4 0 1702 1123 376 0 1758 1327 194 0 1734 1314 0 5 0 1713 1138 407 0 1736 1307 193 0 1730 1341 0
Avg 0 1699 1107 393 0 1683 1261 193 0 1752 1317 0
Table 3: Raw performance evaluation results
121
Appendix D. TEST SCAFFOLDING SOURCE CODE
The latest version of the shared container library test scaffolding is available online at: http://bergeon.francois.perso.neuf.fr/containers/test.zip
D.1 VectorTest.cpp /* * VectorTest.cpp - Test scaffolding vector test * * --- (c) Francois Bergeon 2008 --- * ---- University of Liverpool ---- * * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * * This class implements a vector test * */ #include "VectorTest.h" // Shared container global name static char* _name = "my vector"; // Create/attach vector double VectorTest::create() { // In shared memory if (_shared) { // Instantiate a shared_vector<int> object start_timer(); _s = SharedVectorFactory<int>::create(_name); // Attempt attach if creation failed if (_s == NULL) { start_timer(); _s = SharedVectorFactory<int>::attach(_name); } // Map to a STL vector for testing _c = (std::vector<int>*)_s; } // In local memory else { // Instantiate a vector<int> object start_timer(); _c = new std::vector<int>; } // Error checking
122
if (_c == NULL) return -1.0; return end_timer(); } // Populate vector double VectorTest::populate() { int i; int n = 0; // Start start_timer(); // Exclusive lock if (_shared) _s->write_lock(); // Populate for (i = 0; i < ITER; i++) { // ni = n(i-1) + i n += i; if (_shared) _c->push_back(n); else _c->push_back(n); } // Release exclusive lock if (_shared) _s->write_unlock(); return end_timer(); } // Verify contents double VectorTest::verify() { int i; int n = 0; // Start start_timer(); // Non-exclusive lock if (_shared) _s->read_lock(); // Verify for (i = 0; i < ITER; i++) { // ni = n(i-1) + i n += i; // Fail if no match if ((*_c)[i] != n) { if (_shared) _s->read_unlock(); return -1.0; } } // Release non-exclusive locks if (_shared) _s->read_unlock(); return end_timer(); }
123
// Destroy/detach vector double VectorTest::destroy() { start_timer(); if (_shared) SharedVectorFactory<int>::detach(_name); else delete _c;
return end_timer(); }
Listing 18: VectorTest.cpp
D.2 MapTest.cpp /* * MapTest.cpp - Test scaffolding map test * * --- (c) Francois Bergeon 2008 --- * ---- University of Liverpool ---- * * IMPLEMENTING CONTAINER CLASSES IN SHARED MEMORY * * A dissertation for the completion of a * Master of Science in IT - Software Engineering * * Martha McCormick Dissertation Advisor * * This class implements a vector test * */ #include "MapTest.h" // Shared container global name static char* _name = "my map"; // Create/attach map double MapTest::create() { // In shared memory if (_shared) { // Instantiate a shared_map<string,int> object start_timer(); _s = SharedMapFactory<std::string,int>::create(_name); if (_s == NULL) { start_timer(); _s = SharedMapFactory<std::string,int>::attach(_name); } // Map to a STL map for testing _c = (std::map<std::string,int>*)_s; } else { // Instantiate a map<string,int> object
124
start_timer(); _c = new std::map<std::string,int>; } // Error checking if (_c == NULL) return -1.0; return end_timer(); } // Populate map double MapTest::populate() { int i; char buffer[BUFSIZ]; std::string str; int n = 0; // Start start_timer(); // Exclusive lock if (_shared) _s->write_lock(); // Populate for (i = 0; i < ITER; i++) { // “Char(‘A’ + i%26) & i” char c = 'A' + (i%26); sprintf_s(buffer, BUFSIZ, "%c%d", c, i); str = std::string(buffer); n += i; if (_shared) (*_s)[str] = n; else (*_c)[str] = n; } // Release exclusive lock if (_shared) _s->write_unlock(); return end_timer(); } // Verify contents double MapTest::verify() { int i; char buffer[BUFSIZ]; std::string str; int n1 = 0, n2; // Start start_timer(); // Non-exclusive lock if (_shared) _s->read_lock(); for (i = 0; i < ITER; i++) { // “Char(‘A’ + i%26) & i” char c = 'A' + (i%26); sprintf_s(buffer, BUFSIZ, "%c%d", c, i); str = std::string(buffer); n1 += i;
125
if (_shared) n2 = (*_s)[str]; else n2 = (*_c)[str]; // Fail if no match if (n1 != n2) { if (_shared) _s->read_unlock(); return -1.0; } } // Release non-exclusive locks if (_shared) _s->read_unlock(); return end_timer(); } // Destroy/detach map double MapTest::destroy() { start_timer(); if (_shared) SharedMapFactory<std::string,int>::detach(_name); else delete _c; return end_timer();
}
Listing 19: MapTest.cpp