Logical Regions
Logical regions are the core Legion abstraction for data. Conceptually, a logical region can be thought of as being like a relation or table in a database. In Legion, a logical region is defined as the cross product between an index space and a field space. These terms will be defined in this tutorial. The next example will show how how to access data in logical regions by making physical instances of those regions.
Index Spaces
If a logical region is a table, then the index space for that region is the set of rows in the table. Or to be more precise: rows are implicitly (or explicitly) assigned numbers, or indices, and the index space is the set of these indices.
Indices can be plain numbers, or they can be multi-dimensional (2-D,
3-D, etc.). Thus the set of points in an index space are bounded by
a Rect templated on the dimensionality of the space.
As with Rect and Domain, which were introduced in the index space
example, it can be useful to describe an
index space where the number of dimensions is not known at compile
time. This is simply called IndexSpace. The templated variant is
called IndexSpaceT.
Index spaces are created by invoking one of several overloaded
variants of the create_index_space method of Runtime. This returns
an IndexSpace or IndexSpaceT depending on whether the provided
argument is a Domain or Rect. Examples of these calls can be seen
on lines 20 and 24.
Index spaces are immutable. The set of points contained in an index space cannot be modified after creation. Instead, index spaces can be partitioned to create a number of subspaces that contain subsets of the points of the parent space. This is covered in a subsequent tutorial.
Note that in particular, partitioning can result in an index space that is sparse, i.e. where the set of points contained in the space cannot be accurately described by a bounding rectangle. Sparse index spaces can also be created directly by creating an index space from a set of rectangles or a set of points. These calls are documented in the API but are not demonstrated here. We walso cover how to partition index spaces into subspaces in a subsequent tutorial.
Legion provides a number of API calls that can be used to retrieve
information about an index space, such as the Domain or Rect that
was used to create it. Lines 28 and 31 show examples of
such calls.
Field Spaces
If index spaces describe the rows in a logical region, then field spaces describe the columns.
Field spaces are created using the create_field_space method of
Runtime (line 34).
Field spaces are created empty. Fields in a field space are
dynamically allocated and freed using a FieldAllocator object (line
37). For performance reasons there is a compile-time upper bound
placed on the maximum number of fields that can be allocated in a
single field space. The user has access to this compile-time limit and
can modify it by changing the value assigned to MAX_FIELDS in the
legion_config.h header file (the default is 512). If a program attempts
to exceed this maximum then the Legion runtime will report an error and
exit. There is no limit on the number of field spaces that can be created
in a Legion program and therefore the total number of fields in an
application is unbounded.
Fields are allocated by invoking the allocate_field
method of FieldAllocator (lines 38 and 40). When a
field is allocated the application must specify
the size of the field in bytes.
The allocate_field method will return a FieldID
which is used to name the field. Users may optionally
specify the ID to be associated with the field being
allocated using the second parameter to allocate_field.
If this is done, then it is the responsibility of the
user to ensure that each FieldID is used only once
for a each field space. Legion supports parallel field
allocation in the same field space by different tasks,
but undefined behavior will result if the same FieldID
is allocated in the same field space by two different tasks.
Currently Legion assumes that the data stored by fields is
trivially copyable. If this is not the case, users can also
register custom serialize/deserialize or serdez functors
with the runtime to aid in copying data in certain fields.
The allocate_field method allows users to specify the ID
for one of these functors if the data stored in the field
is not trivally copyable.
Logical Regions
Logical regions are created by passing an index space and a field
space to the create_logical_region method of Runtime (lines 46 and
53). As with index spaces, logical regions can either contain a
dynamic number of dimensions (LogicalRegion) or a templated number
(LogicalRegionT).
Note that, in the example, because both logical regions are created using the same field space, any fields allocated in that field space will be available on both logical regions. Similarly, any logical regions allocated with the same index space will have the same set of points associated with entries in the region.
Every call to create_logical_region will create a new logical region
even when the same index space and field space are passed as arguments
(as seen in lines 60-61). Logical regions are uniquely defined by a
triple consisting of the index space, field space, and region tree
ID. These three values for every logical region can be obtained using
the methods get_index_space (line 48), get_field_space (line 49),
and get_tree_id (line 50). Line 61 demonstrates that the two logical
regions have different region tree IDs.
Resource Reclamation
Index spaces, field spaces, and logical regions are all resources that
use memory. (Although index spaces and field spaces have no memory
directly associated with them, there is still a cost to the metadata
that is allocated.) When applications are done using
resources they should be returned to the runtime.
The destroy_logical_region (line 63-65), destroy_field_space
(line 66), and destroy_index_space (line 67-68) methods
are used to return logical regions, field spaces,
and index spaces to the runtime respectively. Since
Legion operates with a deferred execution model, the
runtime is smart enough to know how to defer deletions
until they are safe to perform. This means that users
do not need to wait for tasks that use logical regions
to finish before issuing deletions commands to
reclaim resources.
If a task neglects to delete a logical region that it created (or index or field space), it will implicitly flow up to the parent task that called it. Thus, it will not be automatically deleted when the task finishes.
FieldAllocator objects are also
resources, but, are similar to Future objects, in that they
are reference counted. When the objects go out of the scope
the destructor is invoked and references are removed. The use of the
explicit C++ scope at line 36 is to ensure that the allocator is
reclaimed as soon as it is done being used.
Next Example: Physical Regions
Previous Example: Hybrid Model
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
#include <cstdio>
#include <cassert>
#include <cstdlib>
#include "legion.h"
using namespace Legion;
enum TaskIDs {
TOP_LEVEL_TASK_ID,
};
enum FieldIDs {
FID_FIELD_A,
FID_FIELD_B,
};
void top_level_task(const Task *task,
const std::vector<PhysicalRegion> ®ions,
Context ctx, Runtime *runtime) {
const Domain domain(DomainPoint(0), DomainPoint(1023));
IndexSpace untyped_is = runtime->create_index_space(ctx, domain);
printf("Created untyped index space %x\n", untyped_is.get_id());
const Rect<1> rect(0,1023);
IndexSpaceT<1> typed_is = runtime->create_index_space(ctx, rect);
printf("Created typed index space %x\n", typed_is.get_id());
{
Domain orig_domain = runtime->get_index_space_domain(ctx, untyped_is);
assert(orig_domain == domain);
Rect<1> orig_rect = runtime->get_index_space_domain(ctx, typed_is);
assert(orig_rect == rect);
}
FieldSpace fs = runtime->create_field_space(ctx);
printf("Created field space field space %x\n", fs.get_id());
{
FieldAllocator allocator = runtime->create_field_allocator(ctx, fs);
FieldID fida = allocator.allocate_field(sizeof(double), FID_FIELD_A);
assert(fida == FID_FIELD_A);
FieldID fidb = allocator.allocate_field(sizeof(int), FID_FIELD_B);
assert(fidb == FID_FIELD_B);
printf("Allocated two fields with Field IDs %d and %d\n", fida, fidb);
}
LogicalRegion untyped_lr =
runtime->create_logical_region(ctx, untyped_is, fs);
printf("Created untyped logical region (%x,%x,%x)\n",
untyped_lr.get_index_space().get_id(),
untyped_lr.get_field_space().get_id(),
untyped_lr.get_tree_id());
LogicalRegionT<1> typed_lr =
runtime->create_logical_region(ctx, typed_is, fs);
printf("Created typed logical region (%x,%x,%x)\n",
typed_lr.get_index_space().get_id(),
typed_lr.get_field_space().get_id(),
typed_lr.get_tree_id());
LogicalRegion no_clone_lr =
runtime->create_logical_region(ctx, typed_is, fs);
assert(typed_lr.get_tree_id() != no_clone_lr.get_tree_id());
runtime->destroy_logical_region(ctx, untyped_lr);
runtime->destroy_logical_region(ctx, typed_lr);
runtime->destroy_logical_region(ctx, no_clone_lr);
runtime->destroy_field_space(ctx, fs);
runtime->destroy_index_space(ctx, untyped_is);
runtime->destroy_index_space(ctx, typed_is);
printf("Successfully cleaned up all of our resources\n");
}
int main(int argc, char **argv) {
Runtime::set_top_level_task_id(TOP_LEVEL_TASK_ID);
{
TaskVariantRegistrar registrar(TOP_LEVEL_TASK_ID, "top_level");
registrar.add_constraint(ProcessorConstraint(Processor::LOC_PROC));
Runtime::preregister_task_variant<top_level_task>(registrar, "top_level");
}
return Runtime::start(argc, argv);
}