Numpy Dtype Error - (structured Array Creation)

I am having some trouble understanding why the following does not work: np.dtype(dict(names='10', formats=np.float64)) I have been struggling with this because I would like to get

Solution 1:

Looks like you are trying to construct a structured array something like:

In [152]: names=['1','2','3','4']
In [153]: formats=[(float,2),(float,3),(float,2),(float,3)]
In [154]: dt=np.dtype({'names':names, 'formats':formats})
In [156]: ds=np.zeros(5, dtype=dt)

In [157]: ds
Out[157]: 
array([([0.0, 0.0], [0.0, 0.0, 0.0], [0.0, 0.0], [0.0, 0.0, 0.0]),
       ([0.0, 0.0], [0.0, 0.0, 0.0], [0.0, 0.0], [0.0, 0.0, 0.0]),
       ([0.0, 0.0], [0.0, 0.0, 0.0], [0.0, 0.0], [0.0, 0.0, 0.0]),
       ([0.0, 0.0], [0.0, 0.0, 0.0], [0.0, 0.0], [0.0, 0.0, 0.0]),
       ([0.0, 0.0], [0.0, 0.0, 0.0], [0.0, 0.0], [0.0, 0.0, 0.0])], 
      dtype=[('1', '<f8', (2,)), ('2', '<f8', (3,)), ('3', '<f8', (2,)), 
           ('4', '<f8', (3,))])
In [159]: ds['1']=np.arange(10).reshape(5,2)
In [160]: ds['2']=np.arange(15).reshape(5,3)

In other words, multiple fields, each with a different number of 'columns' (shape).

Here I create initialize the whole array, and then fill the fields individually. That seems to be the most straight forward way of creating complex structured arrays.

You are trying to build such an array incrementally, starting with one field, and adding new ones with recfunctions.append_fields

In [162]: i=1; 
   ds2 = np.array(np.arange(5),dtype=np.dtype({'names':[str(i)],'formats':[float]}))
In [164]: i+=1;
   ds2=recfunctions.append_fields(base=ds2,names=str(i),dtypes=float,
      data=np.arange(5), usemask=False,asrecarray=False)
In [165]: i+=1;
   ds2=recfunctions.append_fields(base=ds2,names=str(i),dtypes=float,
      data=np.arange(5), usemask=False,asrecarray=False)

In [166]: ds2
Out[166]: 
array(data = [(0.0, 0.0, 0.0) (1.0, 1.0, 1.0) (2.0, 2.0, 2.0) 
    (3.0, 3.0, 3.0) (4.0, 4.0, 4.0)], 
    dtype = [('1', '<f8'), ('2', '<f8'), ('3', '<f8')])

This works when the appended fields all have 1 'column'. With the masking they can even have different numbers of 'rows'. But when I try to vary the internal shape it has problems appending the field. marge_arrays isn't any more successful.

Even if we can get the incremental recfunctions approach to work, it probably will be slower than the initialize-and-fill approach. Even if you don't know the shape of each of the fields at the start, you could collect them all in a dictionary, and assemble the array from that. This kind of structured array isn't any more compact or efficient than a dictionary. It just makes certain styles of data access (across fields) more convenient.

Solution 2:

You must pass a list of values and a list of formats, not a single value and a single format. if you debug the issue, you will find that

type(names)   # result is <type'str'>
type(formats) # result is <type'type'>

It then happens that the dict is initialized to

{'formats': numpy.float64, 'names': '30'}

while each of formats and names should be a list.

EDIT: moreover, notice that formats should be a list of strings, like ['float64','u8'], etc..