I am experimenting with defining dataclasses for containing ctypes objects. I have defined three classes; two of which are used as attributes to the third:
@dataclasses.dataclass
class Point:
x = ctypes.c_int(0)
y = ctypes.c_int(0)
@dataclasses.dataclass
class Point2:
x = ctypes.c_int(0)
y = ctypes.c_int(0)
@dataclasses.dataclass
class TestObj:
point1 = Point()
point2 = Point()
The TestObj class is then passed to a C function inc:
#include <stdio.h>
void inc(int *x, int *y)
{
*x = *x + 1;
*y = *y + 1;
}
via a wrapper:
def inc(test: TestObj):
_inc = lib.inc
_inc.argtypes = [ctypes.POINTER(ctypes.c_int), ctypes.POINTER(ctypes.c_int)]
_inc.restype = None
_inc(test.point1.x,test.point1.y)
When investigating the result of the pass to inc I observe the snippet:
test = TestObj()
print(f"addressof test.point1.x = {ctypes.addressof(test.point1.x)}")
print(f"test.point1.x.value = {test.point1.x.value}")
print(f"addressof test.point1.y = {ctypes.addressof(test.point1.y)}")
print(f"test.point1.y.value = {test.point1.y.value}")
print(f"addressof test.point2.x = {ctypes.addressof(test.point2.x)}")
print(f"test.point2.x.value = {test.point2.x.value}")
print(f"addressof test.point2.y = {ctypes.addressof(test.point2.y)}")
print(f"test.point2.y.value = {test.point2.y.value}")
inc(test)
print("after pass to inc")
print(f"addressof test.point1.x = {ctypes.addressof(test.point1.x)}")
print(f"test.point1.x.value = {test.point1.x.value}")
print(f"addressof test.point1.y = {ctypes.addressof(test.point1.y)}")
print(f"test.point1.y.value = {test.point1.y.value}")
print(f"addressof test.point2.x = {ctypes.addressof(test.point2.x)}")
print(f"test.point2.x.value = {test.point2.x.value}")
print(f"addressof test.point2.y = {ctypes.addressof(test.point2.y)}")
print(f"test.point2.y.value = {test.point2.y.value}")
results in:
addressof test.point1.x = 124872721403536
test.point1.x.value = 0
addressof test.point1.y = 124872719995280
test.point1.y.value = 0
addressof test.point2.x = 124872721403536
test.point2.x.value = 0
addressof test.point2.y = 124872719995280
test.point2.y.value = 0
after pass to inc
addressof test.point1.x = 124872721403536
test.point1.x.value = 1
addressof test.point1.y = 124872719995280
test.point1.y.value = 1
addressof test.point2.x = 124872721403536
test.point2.x.value = 1
addressof test.point2.y = 124872719995280
test.point2.y.value = 1
the memory locations of the subsequent x and y attributes of each Point instance share the same memory location respectively and that the x and y attributes of each Point instance attributes of the TestObj instance are incremented, instead of just the point1 attributes.
If I redefine TestObj as:
@dataclasses.dataclass
class TestObj:
point1 = Point()
point2 = Point2()
and again pass it to the inc wrapper I observe the intended result of:
addressof test.point1.x = 133142487043728
test.point1.x.value = 0
addressof test.point1.y = 133142485635472
test.point1.y.value = 0
addressof test.point2.x = 133142485635728
test.point2.x.value = 0
addressof test.point2.y = 133142485635984
test.point2.y.value = 0
after pass to inc
addressof test.point1.x = 133142487043728
test.point1.x.value = 1
addressof test.point1.y = 133142485635472
test.point1.y.value = 1
addressof test.point2.x = 133142485635728
test.point2.x.value = 0
addressof test.point2.y = 133142485635984
test.point2.y.value = 0
While the x and y attributes of point1 and point2 share the same memory locations repspectively, now only the point1 x and y are incremented as intended.
I am puzzled as to how this is happening, given that in both scenarios the x and y attributes seemingly share the same memory location, but the change in name of the Point attribute provides the intended result. I appreciate any and all help in understanding what is happening.
I am experimenting with defining dataclasses for containing ctypes objects. I have defined three classes; two of which are used as attributes to the third:
@dataclasses.dataclass
class Point:
x = ctypes.c_int(0)
y = ctypes.c_int(0)
@dataclasses.dataclass
class Point2:
x = ctypes.c_int(0)
y = ctypes.c_int(0)
@dataclasses.dataclass
class TestObj:
point1 = Point()
point2 = Point()
The TestObj class is then passed to a C function inc:
#include <stdio.h>
void inc(int *x, int *y)
{
*x = *x + 1;
*y = *y + 1;
}
via a wrapper:
def inc(test: TestObj):
_inc = lib.inc
_inc.argtypes = [ctypes.POINTER(ctypes.c_int), ctypes.POINTER(ctypes.c_int)]
_inc.restype = None
_inc(test.point1.x,test.point1.y)
When investigating the result of the pass to inc I observe the snippet:
test = TestObj()
print(f"addressof test.point1.x = {ctypes.addressof(test.point1.x)}")
print(f"test.point1.x.value = {test.point1.x.value}")
print(f"addressof test.point1.y = {ctypes.addressof(test.point1.y)}")
print(f"test.point1.y.value = {test.point1.y.value}")
print(f"addressof test.point2.x = {ctypes.addressof(test.point2.x)}")
print(f"test.point2.x.value = {test.point2.x.value}")
print(f"addressof test.point2.y = {ctypes.addressof(test.point2.y)}")
print(f"test.point2.y.value = {test.point2.y.value}")
inc(test)
print("after pass to inc")
print(f"addressof test.point1.x = {ctypes.addressof(test.point1.x)}")
print(f"test.point1.x.value = {test.point1.x.value}")
print(f"addressof test.point1.y = {ctypes.addressof(test.point1.y)}")
print(f"test.point1.y.value = {test.point1.y.value}")
print(f"addressof test.point2.x = {ctypes.addressof(test.point2.x)}")
print(f"test.point2.x.value = {test.point2.x.value}")
print(f"addressof test.point2.y = {ctypes.addressof(test.point2.y)}")
print(f"test.point2.y.value = {test.point2.y.value}")
results in:
addressof test.point1.x = 124872721403536
test.point1.x.value = 0
addressof test.point1.y = 124872719995280
test.point1.y.value = 0
addressof test.point2.x = 124872721403536
test.point2.x.value = 0
addressof test.point2.y = 124872719995280
test.point2.y.value = 0
after pass to inc
addressof test.point1.x = 124872721403536
test.point1.x.value = 1
addressof test.point1.y = 124872719995280
test.point1.y.value = 1
addressof test.point2.x = 124872721403536
test.point2.x.value = 1
addressof test.point2.y = 124872719995280
test.point2.y.value = 1
the memory locations of the subsequent x and y attributes of each Point instance share the same memory location respectively and that the x and y attributes of each Point instance attributes of the TestObj instance are incremented, instead of just the point1 attributes.
If I redefine TestObj as:
@dataclasses.dataclass
class TestObj:
point1 = Point()
point2 = Point2()
and again pass it to the inc wrapper I observe the intended result of:
addressof test.point1.x = 133142487043728
test.point1.x.value = 0
addressof test.point1.y = 133142485635472
test.point1.y.value = 0
addressof test.point2.x = 133142485635728
test.point2.x.value = 0
addressof test.point2.y = 133142485635984
test.point2.y.value = 0
after pass to inc
addressof test.point1.x = 133142487043728
test.point1.x.value = 1
addressof test.point1.y = 133142485635472
test.point1.y.value = 1
addressof test.point2.x = 133142485635728
test.point2.x.value = 0
addressof test.point2.y = 133142485635984
test.point2.y.value = 0
While the x and y attributes of point1 and point2 share the same memory locations repspectively, now only the point1 x and y are incremented as intended.
I am puzzled as to how this is happening, given that in both scenarios the x and y attributes seemingly share the same memory location, but the change in name of the Point attribute provides the intended result. I appreciate any and all help in understanding what is happening.
this is not the intended use of dataclasses - you were just lucky (or unlucky) the Python side of the code didn't error to startwith:
Upon doing
@dataclasses.dataclass
class Point:
x = ctypes.c_int(0)
y = ctypes.c_int(0)
You are not indicating that x and y should be instances of ctypes.c_int - you are creating two actual instances of c_int with a definite memory address, as class attributes for Point.
The dataclass decorator will look at the values, check there are no annotations, and simply annotating that - when a new instance is initialized, the received values should be instances of c_int
When you instantiate Point without arguments, the dataclass will just leave use the existing, class wide, instances of c_int as the default values.
It would work if each time you'd instantiate a point (and therefore a test object), you'd pass a new instance of c_int (four new c_ints for each instance of TestOBJ - in other words: it is not just meant to work this way.
What will create you new c level integers, convert automatically from Python ints to C values, and have nice defaults are c_types.Structure classes you really, like in really should use those instead of dataclasses for interoperating with ctypes.
It is easy to build a @dataclass like decorator to build Structure classes with the proper _fields_ parameter set, if you prefer that syntax - but dataclass is not that decorator.
This should work for simple cases instead:
def datastructure(cls):
_fields_ = [(k, v) for k, v in cls.__dict__.items() if not k.startswith("_")]
new_cls = type(cls.__name__, (ctypes.Structure,), {"_fields_": _fields_})
return new_cls
ANd on the repl:
In [5]: @datastructure
...: class Point:
...: x = ctypes.c_int
...: y = ctypes.c_int
...:
In [6]: p = Point(10, 20)
In [7]: p.x
Out[7]: 10
In [8]: p.y
Out[8]: 20
I don't think your use of dataclass is correct. First of all, in every documented example, variable declarations include type information. I checked the standard library docs and PEP 557. If I overlooked something, I'm sure someone will correct me.
All the simple examples follow this pattern (there are a lot of variations):
@dataclass
class C:
x: int = 0
This creates a class C and with a constructor __init__(self, c: int=0).
What you have done is to omit the type definition:
@dataclass
class C:
x = 0
I don't know why this works at all, but it does. It creates a class C with zero-argument constructor __init__(self). When I instantiate an instance of C, I see that the variable x is initialized to 0 in the class __dict__; the instance __dict__ is empty. This explains why the address of p.x is always the same no matter how many Points() you create: because each p.x is in fact the same object, which lives in the class __dict__.
As your Point class stands, it has only a zero-argument constructor. This seems pretty useless to me. To obtain an instance of Point that doesn't have x=0 and y=0, you must explicitly set those variables after the object is created. In other words, to make a Point representing (1,1) you need 3 lines of code:
p = Point()
p.x = ctypes.c_int(1)
p.y = ctypes.c_int(1)
You could write your point class like this:
@dataclass
class Point:
x: ctypes.c_int = ctypes.c_int(0)
y: ctypes.c_int = ctypes.c_int(0)
This has a constructor
__init__(self, x: ctypes.cint = ctypes.c_int(0),
y: ctypes.cint = ctypes.c_int(0))
which at least allows you to write this:
p11 = Point(ctypes.c_int(1), ctypes.c_int(1))
but that still is clumsy.
I would prefer dropping the whole dataclass concept. What you want, I think, is a constructor that takes integers for x and y and converts them to c_int(s).
class Point:
def __init__(self, x: int = 0, y: int = 0) -> None:
self.cx = ctypes.c_int(x)
self.cy = ctypes.c_int(y)
p = Point() # 0, 0
p11 = Point(1, 1) # 1, 1
__init__for dataclasses, but I now see that this is incorrect. – frank Commented Mar 4 at 10:59