Field3D
Field3DInputFileHDF5 Class Reference

Provides reading of .f3d (internally, hdf5) files. More...

#include <Field3DFileHDF5.h>

Inheritance diagram for Field3DInputFileHDF5:
Field3DFileHDF5Base MetadataCallback

Public Member Functions

bool open (const std::string &filename)
 Opens the given file. More...
 
bool readGroupMembership (GroupMembershipMap &gpMembershipMap)
 Read the group membership for the partitions. More...
 
Constructors & destructor
 Field3DInputFileHDF5 ()
 
virtual ~Field3DInputFileHDF5 ()
 
Reading layers from disk
template<class Data_T >
Field< Data_T >::Vec readScalarLayers (const std::string &layerName=std::string("")) const
 Retrieves all the layers of scalar type and maintains their on-disk data types. More...
 
template<class Data_T >
Field< Data_T >::Vec readScalarLayers (const std::string &partitionName, const std::string &layerName) const
 This one allows the allows the partitionName to be passed in. More...
 
template<class Data_T >
Field< FIELD3D_VEC3_T< Data_T > >::Vec readVectorLayers (const std::string &layerName=std::string("")) const
 Retrieves all the layers of vector type and maintains their on-disk data types. More...
 
template<class Data_T >
Field< FIELD3D_VEC3_T< Data_T > >::Vec readVectorLayers (const std::string &partitionName, const std::string &layerName) const
 This version allows you to pass in the partition name. More...
 
template<template< typename T > class Field_T, class Data_T >
Field_T< Data_T >::Vec readScalarLayersAs (const std::string &layerName=std::string("")) const
 Retrieves all layers for all partitions. Converts it to the given template type if needed. More...
 
template<template< typename T > class Field_T, class Data_T >
Field_T< Data_T >::Vec readScalarLayersAs (const std::string &partitionName, const std::string &layerName) const
 Retrieves a layers given their and its parent partition's name. Converts it to the given template type if needed. More...
 
template<template< typename T > class Field_T, class Data_T >
Field_T< FIELD3D_VEC3_T< Data_T > >::Vec readVectorLayersAs (const std::string &layerName=std::string("")) const
 Retrieves a layers for all partitions. Converts it to the given template type if needed. More...
 
template<template< typename T > class Field_T, class Data_T >
Field_T< FIELD3D_VEC3_T< Data_T > >::Vec readVectorLayersAs (const std::string &partitionName, const std::string &layerName) const
 Retrieves a layers given their and its parent partition's name. Converts it to the given template type if needed. More...
 
Reading proxy data from disk
template<class Data_T >
EmptyField< Data_T >::Vec readProxyLayer (const std::string &partitionName, const std::string &layerName, bool isVectorLayer) const
 Retrieves a proxy version (EmptyField) of each layer . More...
 
template<class Data_T >
EmptyField< Data_T >::Ptr readProxyLayer (hid_t location, const std::string &name, const std::string &attribute, FieldMapping::Ptr mapping) const
 Retrieves a proxy version (EmptyField) from a given HDF5 location. More...
 
template<class Data_T >
EmptyField< Data_T >::Vec readProxyScalarLayers (const std::string &name=std::string("")) const
 Retrieves a proxy version (EmptyField) of each scalar layer. More...
 
template<class Data_T >
EmptyField< Data_T >::Vec readProxyVectorLayers (const std::string &name=std::string("")) const
 Retrieves a proxy version (EmptyField) of each vector layer. More...
 
Internal utility methods
herr_t parsePartition (hid_t loc_id, const std::string partitionName)
 Gets called from parsePartitions. Not intended for any other use. More...
 
herr_t parseLayer (hid_t loc_id, const std::string &partitionName, const std::string &layerName)
 Gets called from parsePartitions. Not intended for any other use. More...
 
- Public Member Functions inherited from Field3DFileHDF5Base
void clear ()
 Clear the data structures and close the file. More...
 
bool close ()
 Closes the file. No need to call this unless you specifically want to close the file early. It will close once the FileHDF5 object goes out of scope. More...
 
FieldMetadatametadata ()
 accessor to the m_metadata class More...
 
const FieldMetadatametadata () const
 Read only access to the m_metadata class. More...
 
virtual void metadataHasChanged (const std::string &)
 This function should implemented by concrete classes to
get the callback when metadata changes. More...
 
 Field3DFileHDF5Base ()
 
virtual ~Field3DFileHDF5Base ()=0
 Pure virtual destructor to ensure we never instantiate this class. More...
 
void getPartitionNames (std::vector< std::string > &names) const
 Gets the names of all the partitions in the file. More...
 
void getScalarLayerNames (std::vector< std::string > &names, const std::string &partitionName) const
 Gets the names of all the scalar layers in a given partition. More...
 
void getVectorLayerNames (std::vector< std::string > &names, const std::string &partitionName) const
 Gets the names of all the vector layers in a given partition. More...
 
FileHDF5::Partition::Ptr getPartition (const std::string &partitionName) const
 Returns a pointer to the given partition. More...
 
std::string intPartitionName (const std::string &partitionName, const std::string &layerName, FieldRes::Ptr field)
 Returns a unique partition name given the requested name. This ensures that partitions with matching mappings get the same name but each subsequent differing mapping gets a new, separate name. More...
 
std::string removeUniqueId (const std::string &partitionName) const
 Strips any unique identifiers from the partition name and returns the original name. More...
 
void addGroupMembership (const GroupMembershipMap &groupMembers)
 Add to the group membership. More...
 
void printHierarchy () const
 

Private Member Functions

template<class Data_T >
Field< Data_T >::Ptr readLayer (const std::string &intPartitionName, const std::string &layerName, bool isVectorLayer) const
 This call does the actual reading of a layer. Notice that it expects a unique -internal- partition name. More...
 
bool readMetadata (hid_t metadata_id)
 Read global metadata for this file. More...
 
bool readMetadata (hid_t metadata_id, FieldBase::Ptr field) const
 Read metadata for this layer. More...
 
bool readPartitionAndLayerInfo ()
 Sets up all the partitions and layers, but does not load any data. More...
 
template<class Data_T >
Field< Data_T >::Ptr readScalarLayer (const std::string &intPartitionName, const std::string &layerName) const
 Retrieves a single layer given its and its parent partition's name. Maintains the on-disk data types. More...
 
template<class Data_T >
Field< FIELD3D_VEC3_T< Data_T > >::Ptr readVectorLayer (const std::string &intPartitionName, const std::string &layerName) const
 Retrieves a single layer given its and its parent partition's name. Maintains the on-disk data types. More...
 

Private Attributes

std::string m_filename
 Filename, only to be set by open(). More...
 

Friends

class Field3DInputFile
 
class Field3DOutputFile
 

Additional Inherited Members

- Public Types inherited from Field3DFileHDF5Base
typedef std::map< std::string, std::string > GroupMembershipMap
 
- Protected Types inherited from Field3DFileHDF5Base
typedef std::map< std::string, int > PartitionCountMap
 
typedef std::vector< FileHDF5::Partition::PtrPartitionList
 
- Protected Member Functions inherited from Field3DFileHDF5Base
void closeInternal ()
 Closes the file if open. More...
 
FileHDF5::Partition::Ptr partition (const std::string &partitionName)
 Returns a pointer to the given partition. More...
 
FileHDF5::Partition::Ptr partition (const std::string &partitionName) const
 Returns a pointer to the given partition. More...
 
void getIntPartitionNames (std::vector< std::string > &names) const
 Gets the names of all the -internal- partitions in the file. More...
 
void getIntScalarLayerNames (std::vector< std::string > &names, const std::string &intPartitionName) const
 Gets the names of all the scalar layers in a given partition, but assumes that partition name is the -internal- partition name. More...
 
void getIntVectorLayerNames (std::vector< std::string > &names, const std::string &intPartitionName) const
 Gets the names of all the vector layers in a given partition, but assumes that partition name is the -internal- partition name. More...
 
int numIntPartitions (const std::string &partitionName) const
 Returns the number of internal partitions for a given partition name. More...
 
std::string makeIntPartitionName (const std::string &partitionsName, int i) const
 Makes an internal partition name given the external partition name. Effectively just tacks on .X to the name, where X is the number. More...
 
- Protected Attributes inherited from Field3DFileHDF5Base
hid_t m_file
 The hdf5 id of the current file. Will be -1 if no file is open. More...
 
GroupMembershipMap m_groupMembership
 Keeps track of group membership for each layer of partition name. The key is the "group" and the value is a space separated list of "partitionName.0:Layer1 partitionName.1:Layer0 ..."
More...
 
std::vector< LayerInfom_layerInfo
 This stores layer info. More...
 
FieldMetadata m_metadata
 metadata More...
 
PartitionCountMap m_partitionCount
 Contains a counter for each partition name. This is used to keep multiple fields with the same name unique in the file. More...
 
std::vector< std::string > m_partitionNames
 This stores partition names. More...
 
PartitionList m_partitions
 Vector of partitions. More...
 

Detailed Description

Provides reading of .f3d (internally, hdf5) files.

Refer to using_files for examples of how to use this in your code.

Note
We distinguish between scalar and vector layers even though both are templated. A scalarField<float> layer is interchangeable with a scalarField<double> (conceptually) but not with a scalar<V3f>, and thus not with vectorField<float>.

Definition at line 437 of file Field3DFileHDF5.h.

Constructor & Destructor Documentation

◆ Field3DInputFileHDF5()

Field3DInputFileHDF5::Field3DInputFileHDF5 ( )

Definition at line 572 of file Field3DFileHDF5.cpp.

573 {
574  // Empty
575 }

◆ ~Field3DInputFileHDF5()

Field3DInputFileHDF5::~Field3DInputFileHDF5 ( )
virtual

Definition at line 579 of file Field3DFileHDF5.cpp.

580 {
581  clear();
582 }

References Field3DFileHDF5Base::clear().

Member Function Documentation

◆ readScalarLayers() [1/2]

template<class Data_T >
Field< Data_T >::Vec Field3DInputFileHDF5::readScalarLayers ( const std::string &  layerName = std::string("")) const

Retrieves all the layers of scalar type and maintains their on-disk data types.

Parameters
layerNameIf a string is passed in, only layers of that name will be read from disk.

Definition at line 910 of file Field3DFileHDF5.h.

911 {
912  using namespace std;
913 
914  typedef typename Field<Data_T>::Ptr FieldPtr;
915  typedef typename Field<Data_T>::Vec FieldList;
916 
917  FieldList ret;
918  std::vector<std::string> parts;
919  getIntPartitionNames(parts);
920 
921  for (vector<string>::iterator p = parts.begin(); p != parts.end(); ++p) {
922  std::vector<std::string> layers;
923  getIntScalarLayerNames(layers, *p);
924  for (vector<string>::iterator l = layers.begin(); l != layers.end(); ++l) {
925  // Only read if it matches the name
926  if ((name.length() == 0) || (*l == name)) {
927  FieldPtr mf = readScalarLayer<Data_T>(*p, *l);
928  if (mf) {
929  ret.push_back(mf);
930  }
931  }
932  }
933  }
934 
935  return ret;
936 }

References Field3DFileHDF5Base::getIntPartitionNames(), and Field3DFileHDF5Base::getIntScalarLayerNames().

◆ readScalarLayers() [2/2]

template<class Data_T >
Field< Data_T >::Vec Field3DInputFileHDF5::readScalarLayers ( const std::string &  partitionName,
const std::string &  layerName 
) const

This one allows the allows the partitionName to be passed in.

Definition at line 942 of file Field3DFileHDF5.h.

944 {
945  using namespace std;
946 
947  typedef typename Field<Data_T>::Ptr FieldPtr;
948  typedef typename Field<Data_T>::Vec FieldList;
949 
950  FieldList ret;
951 
952  if ((layerName.length() == 0) || (partitionName.length() == 0))
953  return ret;
954 
955  std::vector<std::string> parts;
956  getIntPartitionNames(parts);
957 
958  for (vector<string>::iterator p = parts.begin(); p != parts.end(); ++p) {
959  std::vector<std::string> layers;
960  getIntScalarLayerNames(layers, *p);
961  if (removeUniqueId(*p) == partitionName) {
962  for (vector<string>::iterator l = layers.begin();
963  l != layers.end(); ++l) {
964  // Only read if it matches the name
965  if (*l == layerName) {
966  FieldPtr mf = readScalarLayer<Data_T>(*p, *l);
967  if (mf)
968  ret.push_back(mf);
969  }
970  }
971  }
972  }
973 
974  return ret;
975 }

References Field3DFileHDF5Base::getIntPartitionNames(), Field3DFileHDF5Base::getIntScalarLayerNames(), and Field3DFileHDF5Base::removeUniqueId().

◆ readVectorLayers() [1/2]

template<class Data_T >
Field< FIELD3D_VEC3_T< Data_T > >::Vec Field3DInputFileHDF5::readVectorLayers ( const std::string &  layerName = std::string("")) const

Retrieves all the layers of vector type and maintains their on-disk data types.

Parameters
layerNameIf a string is passed in, only layers of that name will be read from disk.

Definition at line 981 of file Field3DFileHDF5.h.

982 {
983  using namespace std;
984 
985  typedef typename Field<FIELD3D_VEC3_T<Data_T> >::Ptr FieldPtr;
986  typedef typename Field<FIELD3D_VEC3_T<Data_T> >::Vec FieldList;
987 
988  FieldList ret;
989 
990  std::vector<std::string> parts;
991  getIntPartitionNames(parts);
992 
993  for (vector<string>::iterator p = parts.begin(); p != parts.end(); ++p) {
994  std::vector<std::string> layers;
995  getIntVectorLayerNames(layers, *p);
996  for (vector<string>::iterator l = layers.begin(); l != layers.end(); ++l) {
997  // Only read if it matches the name
998  if ((name.length() == 0) || (*l == name)) {
999  FieldPtr mf = readVectorLayer<Data_T>(*p, *l);
1000  if (mf)
1001  ret.push_back(mf);
1002  }
1003  }
1004  }
1005 
1006  return ret;
1007 }

References Field3DFileHDF5Base::getIntPartitionNames(), and Field3DFileHDF5Base::getIntVectorLayerNames().

◆ readVectorLayers() [2/2]

template<class Data_T >
Field< FIELD3D_VEC3_T< Data_T > >::Vec Field3DInputFileHDF5::readVectorLayers ( const std::string &  partitionName,
const std::string &  layerName 
) const

This version allows you to pass in the partition name.

Definition at line 1013 of file Field3DFileHDF5.h.

1015 {
1016  using namespace std;
1017 
1018  typedef typename Field<FIELD3D_VEC3_T<Data_T> >::Ptr FieldPtr;
1019  typedef typename Field<FIELD3D_VEC3_T<Data_T> >::Vec FieldList;
1020 
1021  FieldList ret;
1022 
1023  if ((layerName.length() == 0) || (partitionName.length() == 0))
1024  return ret;
1025 
1026  std::vector<std::string> parts;
1027  getIntPartitionNames(parts);
1028 
1029  for (vector<string>::iterator p = parts.begin(); p != parts.end(); ++p) {
1030  std::vector<std::string> layers;
1031  getIntVectorLayerNames(layers, *p);
1032  if (removeUniqueId(*p) == partitionName) {
1033  for (vector<string>::iterator l = layers.begin();
1034  l != layers.end(); ++l) {
1035  // Only read if it matches the name
1036  if (*l == layerName) {
1037  FieldPtr mf = readVectorLayer<Data_T>(*p, *l);
1038  if (mf)
1039  ret.push_back(mf);
1040  }
1041  }
1042  }
1043  }
1044 
1045  return ret;
1046 }

References Field3DFileHDF5Base::getIntPartitionNames(), Field3DFileHDF5Base::getIntVectorLayerNames(), and Field3DFileHDF5Base::removeUniqueId().

◆ readScalarLayersAs() [1/2]

template<template< typename T > class Field_T, class Data_T >
Field_T<Data_T>::Vec Field3DInputFileHDF5::readScalarLayersAs ( const std::string &  layerName = std::string("")) const
inline

Retrieves all layers for all partitions. Converts it to the given template type if needed.

Definition at line 491 of file Field3DFileHDF5.h.

492  {
493  typedef typename Field<Data_T>::Vec FieldList;
494  typedef typename Field_T<Data_T>::Vec TypedFieldList;
495 
496  // First, read the layers as-is
497  FieldList originals;
498  originals = readScalarLayers<Data_T>(layerName);
499 
500  // Loop over fields, converting if needed
501  TypedFieldList output;
502  typename FieldList::iterator i = originals.begin();
503  for (; i != originals.end(); ++i) {
504  typename Field_T<Data_T>::Ptr targetField;
505  targetField = field_dynamic_cast<Field_T<Data_T> >(*i);
506  if (targetField) {
507  output.push_back(targetField);
508  } else {
509  typename Field_T<Data_T>::Ptr newTarget(new Field_T<Data_T>);
510  newTarget->name = (*i)->name;
511  newTarget->attribute = (*i)->attribute;
512  newTarget->copyMetadata(**i);
513  newTarget->copyFrom(*i);
514  output.push_back(newTarget);
515  }
516  }
517 
518  return output;
519  }

◆ readScalarLayersAs() [2/2]

template<template< typename T > class Field_T, class Data_T >
Field_T<Data_T>::Vec Field3DInputFileHDF5::readScalarLayersAs ( const std::string &  partitionName,
const std::string &  layerName 
) const
inline

Retrieves a layers given their and its parent partition's name. Converts it to the given template type if needed.

Definition at line 525 of file Field3DFileHDF5.h.

527  {
528  typedef typename Field<Data_T>::Vec FieldList;
529  typedef typename Field_T<Data_T>::Vec TypedFieldList;
530 
531  // First, read the layers as-is
532  FieldList originals;
533  originals = readScalarLayers<Data_T>(partitionName, layerName);
534 
535  // Loop over fields, converting if needed
536  TypedFieldList output;
537  typename FieldList::iterator i = originals.begin();
538  for (; i != originals.end(); ++i) {
539  typename Field_T<Data_T>::Ptr targetField;
540  targetField = field_dynamic_cast<Field_T<Data_T> >(*i);
541  if (targetField) {
542  output.push_back(targetField);
543  } else {
544  typename Field_T<Data_T>::Ptr newTarget(new Field_T<Data_T>);
545  newTarget->name = (*i)->name;
546  newTarget->attribute = (*i)->attribute;
547  newTarget->copyMetadata(**i);
548  newTarget->copyFrom(*i);
549  output.push_back(newTarget);
550  }
551  }
552 
553  return output;
554  }

◆ readVectorLayersAs() [1/2]

template<template< typename T > class Field_T, class Data_T >
Field_T<FIELD3D_VEC3_T<Data_T> >::Vec Field3DInputFileHDF5::readVectorLayersAs ( const std::string &  layerName = std::string("")) const
inline

Retrieves a layers for all partitions. Converts it to the given template type if needed.

Definition at line 560 of file Field3DFileHDF5.h.

561  {
562  typedef Field_T<FIELD3D_VEC3_T<Data_T> > TypedVField;
563  typedef typename Field<FIELD3D_VEC3_T<Data_T> >::Vec FieldList;
564  typedef typename Field_T<FIELD3D_VEC3_T<Data_T> >::Vec TypedFieldList;
565 
566  // First, read the layers as-is
567  FieldList originals;
568  originals = readVectorLayers<Data_T>(layerName);
569 
570  // Loop over fields, converting if needed
571  TypedFieldList output;
572  typename FieldList::iterator i = originals.begin();
573  for (; i != originals.end(); ++i) {
574  typename TypedVField::Ptr targetField;
575  targetField = field_dynamic_cast<TypedVField>(*i);
576  if (targetField) {
577  output.push_back(targetField);
578  } else {
579  typename TypedVField::Ptr newTarget(new TypedVField);
580  newTarget->name = (*i)->name;
581  newTarget->attribute = (*i)->attribute;
582  newTarget->copyMetadata(**i);
583  newTarget->copyFrom(*i);
584  output.push_back(newTarget);
585  }
586  }
587 
588  return output;
589  }

◆ readVectorLayersAs() [2/2]

template<template< typename T > class Field_T, class Data_T >
Field_T<FIELD3D_VEC3_T<Data_T> >::Vec Field3DInputFileHDF5::readVectorLayersAs ( const std::string &  partitionName,
const std::string &  layerName 
) const
inline

Retrieves a layers given their and its parent partition's name. Converts it to the given template type if needed.

Definition at line 595 of file Field3DFileHDF5.h.

597  {
598  typedef Field_T<FIELD3D_VEC3_T<Data_T> > TypedVField;
599  typedef typename Field<FIELD3D_VEC3_T<Data_T> >::Vec FieldList;
600  typedef typename Field_T<FIELD3D_VEC3_T<Data_T> >::Vec TypedFieldList;
601 
602  // First, read the layers as-is
603  FieldList originals;
604  originals = readVectorLayers<Data_T>(partitionName, layerName);
605 
606  // Loop over fields, converting if needed
607  TypedFieldList output;
608  typename FieldList::iterator i = originals.begin();
609  for (; i != originals.end(); ++i) {
610  typename TypedVField::Ptr targetField;
611  targetField = field_dynamic_cast<TypedVField>(*i);
612  if (targetField) {
613  output.push_back(targetField);
614  } else {
615  typename TypedVField::Ptr newTarget(new TypedVField);
616  newTarget->name = (*i)->name;
617  newTarget->attribute = (*i)->attribute;
618  newTarget->copyMetadata(**i);
619  newTarget->copyFrom(*i);
620  output.push_back(newTarget);
621  }
622  }
623 
624  return output;
625  }

◆ readProxyLayer() [1/2]

template<class Data_T >
EmptyField< Data_T >::Vec Field3DInputFileHDF5::readProxyLayer ( const std::string &  partitionName,
const std::string &  layerName,
bool  isVectorLayer 
) const

Retrieves a proxy version (EmptyField) of each layer .

Note
Although the call is templated, all fields are read, regardless of bit depth.
Parameters
nameIf a string is passed in, only layers of that name will be read from disk.

Definition at line 1157 of file Field3DFileHDF5.h.

1160 {
1161  using namespace boost;
1162  using namespace std;
1163  using namespace Hdf5Util;
1164 
1165  GlobalLock lock(g_hdf5Mutex);
1166 
1167  // Instantiate a null pointer for easier code reading
1168  typename EmptyField<Data_T>::Vec emptyList, output;
1169 
1170  if ((layerName.length() == 0) || (partitionName.length() == 0))
1171  return emptyList;
1172 
1173  std::vector<std::string> parts, layers;
1174  getIntPartitionNames(parts);
1175 
1176  bool foundPartition = false;
1177 
1178  for (vector<string>::iterator p = parts.begin(); p != parts.end(); ++p) {
1179  if (removeUniqueId(*p) == partitionName) {
1180  foundPartition = true;
1181  if (isVectorLayer) {
1182  getIntVectorLayerNames(layers, *p);
1183  } else {
1184  getIntScalarLayerNames(layers, *p);
1185  }
1186  for (vector<string>::iterator l = layers.begin();
1187  l != layers.end(); ++l) {
1188  if (*l == layerName) {
1189  // Find the partition
1191  if (!part) {
1192  Msg::print(Msg::SevWarning, "Couldn't find partition: " + *p);
1193  return emptyList;
1194  }
1195  // Find the layer
1196  const FileHDF5::Layer *layer;
1197  if (isVectorLayer)
1198  layer = part->vectorLayer(layerName);
1199  else
1200  layer = part->scalarLayer(layerName);
1201  if (!layer) {
1202  Msg::print(Msg::SevWarning, "Couldn't find layer: " + layerName);
1203  return emptyList;
1204  }
1205  // Open the layer group
1206  string layerPath = layer->parent + "/" + layer->name;
1207  H5ScopedGopen layerGroup(m_file, layerPath.c_str());
1208  if (layerGroup.id() < 0) {
1209  Msg::print(Msg::SevWarning, "Couldn't find layer group "
1210  + layerName + " in .f3d file ");
1211  return emptyList;
1212  }
1213 
1214  // Make the proxy representation
1215  typename EmptyField<Data_T>::Ptr field =
1216  readProxyLayer<Data_T>(layerGroup, partitionName, layerName,
1217  part->mapping);
1218 
1219  // Read MIPField's number of mip levels
1220  int numLevels = 0;
1221  H5ScopedGopen mipGroup(layerGroup, "mip_levels");
1222  if (mipGroup.id() >= 0)
1223  readAttribute(mipGroup, "levels", 1, numLevels);
1224  field->metadata().setIntMetadata("mip_levels", numLevels);
1225 
1226  // Add field to output
1227  output.push_back(field);
1228  }
1229  }
1230  }
1231  }
1232 
1233  if (!foundPartition) {
1234  Msg::print(Msg::SevWarning, "Couldn't find partition: " + partitionName);
1235  return emptyList;
1236  }
1237 
1238  return output;
1239 }

References g_hdf5Mutex, Field3DFileHDF5Base::getIntPartitionNames(), Field3DFileHDF5Base::getIntScalarLayerNames(), Field3DFileHDF5Base::getIntVectorLayerNames(), Hdf5Util::H5Base::id(), Field3DFileHDF5Base::m_file, FieldBase::metadata(), FileHDF5::Layer::name, FileHDF5::Layer::parent, Field3DFileHDF5Base::partition(), Msg::print(), Hdf5Util::readAttribute(), Field3DFileHDF5Base::removeUniqueId(), FieldMetadata::setIntMetadata(), and Msg::SevWarning.

◆ readProxyLayer() [2/2]

template<class Data_T >
EmptyField< Data_T >::Ptr Field3DInputFileHDF5::readProxyLayer ( hid_t  location,
const std::string &  name,
const std::string &  attribute,
FieldMapping::Ptr  mapping 
) const

Retrieves a proxy version (EmptyField) from a given HDF5 location.

Note
Although the call is templated, all fields are read, regardless of bit depth.
Parameters
locationHDF5 file location

Definition at line 1245 of file Field3DFileHDF5.h.

1249 {
1250  using namespace boost;
1251  using namespace std;
1252  using namespace Hdf5Util;
1253 
1254  typename EmptyField<Data_T>::Ptr null;
1255 
1256  GlobalLock lock(g_hdf5Mutex);
1257 
1258  // Read the extents and data window
1259  Box3i extents, dataW;
1260  if (!readAttribute(location, "extents", 6, extents.min.x)) {
1261  return null;
1262  }
1263  if (!readAttribute(location, "data_window", 6, dataW.min.x)) {
1264  return null;
1265  }
1266 
1267  // Construct the field and load the data
1268  typename EmptyField<Data_T>::Ptr field(new EmptyField<Data_T>);
1269  field->setSize(extents, dataW);
1270 
1271  // Read the metadata
1272  H5ScopedGopen metadataGroup(location, "metadata");
1273  if (metadataGroup.id() > 0) {
1274  readMetadata(metadataGroup.id(), field);
1275  }
1276 
1277  // Set field properties
1278  field->name = name;
1279  field->attribute = attribute;
1280  field->setMapping(mapping);
1281 
1282  return field;
1283 }

References FieldBase::attribute, g_hdf5Mutex, Hdf5Util::H5Base::id(), FieldBase::name, Hdf5Util::readAttribute(), readMetadata(), FieldRes::setMapping(), and ResizableField< Data_T >::setSize().

◆ readProxyScalarLayers()

template<class Data_T >
EmptyField< Data_T >::Vec Field3DInputFileHDF5::readProxyScalarLayers ( const std::string &  name = std::string("")) const

Retrieves a proxy version (EmptyField) of each scalar layer.

Note
Although the call is templated, all fields are read, regardless of bit depth.
Parameters
nameIf a string is passed in, only layers of that name will be read from disk.

Definition at line 1289 of file Field3DFileHDF5.h.

1290 {
1291  using namespace std;
1292 
1293  typedef typename EmptyField<Data_T>::Ptr FieldPtr;
1294  typedef std::vector<FieldPtr> FieldList;
1295 
1296  FieldList ret;
1297 
1298  std::vector<std::string> parts;
1299  getPartitionNames(parts);
1300 
1301  for (vector<string>::iterator p = parts.begin(); p != parts.end(); ++p) {
1302  std::vector<std::string> layers;
1303  getScalarLayerNames(layers, *p);
1304  for (vector<string>::iterator l = layers.begin(); l != layers.end(); ++l) {
1305  // Only read if it matches the name
1306  if ((name.length() == 0) || (*l == name)) {
1307  FieldList f = readProxyLayer<Data_T>(*p, *l, false);
1308  for (typename FieldList::iterator i = f.begin(); i != f.end(); ++i) {
1309  if (*i) {
1310  ret.push_back(*i);
1311  }
1312  }
1313  }
1314  }
1315  }
1316 
1317  return ret;
1318 }

References Field3DFileHDF5Base::getPartitionNames(), and Field3DFileHDF5Base::getScalarLayerNames().

◆ readProxyVectorLayers()

template<class Data_T >
EmptyField< Data_T >::Vec Field3DInputFileHDF5::readProxyVectorLayers ( const std::string &  name = std::string("")) const

Retrieves a proxy version (EmptyField) of each vector layer.

Note
Although the call is templated, all fields are read, regardless of bit depth.
Parameters
nameIf a string is passed in, only layers of that name will be read from disk.

Definition at line 1324 of file Field3DFileHDF5.h.

1325 {
1326  using namespace std;
1327 
1328  typedef typename EmptyField<Data_T>::Ptr FieldPtr;
1329  typedef std::vector<FieldPtr> FieldList;
1330 
1331  FieldList ret;
1332 
1333  std::vector<std::string> parts;
1334  getPartitionNames(parts);
1335 
1336  for (vector<string>::iterator p = parts.begin(); p != parts.end(); ++p) {
1337  std::vector<std::string> layers;
1338  getVectorLayerNames(layers, *p);
1339  for (vector<string>::iterator l = layers.begin(); l != layers.end(); ++l) {
1340  // Only read if it matches the name
1341  if ((name.length() == 0) || (*l == name)) {
1342  FieldList f = readProxyLayer<Data_T>(*p, *l, true);
1343  for (typename FieldList::iterator i = f.begin(); i != f.end(); ++i) {
1344  if (*i) {
1345  ret.push_back(*i);
1346  }
1347  }
1348  }
1349  }
1350  }
1351 
1352  return ret;
1353 }

References Field3DFileHDF5Base::getPartitionNames(), and Field3DFileHDF5Base::getVectorLayerNames().

◆ open()

bool Field3DInputFileHDF5::open ( const std::string &  filename)

Opens the given file.

Returns
Whether successful

Definition at line 586 of file Field3DFileHDF5.cpp.

587 {
588  GlobalLock lock(g_hdf5Mutex);
589 
590  clear();
591 
592  bool success = true;
593 
594  // Record filename
595  m_filename = filename;
596 
597  try {
598 
599  string version;
600 
601  // Throws exceptions if the file doesn't exist.
602  // This was added because H5Fopen prints out a lot of junk
603  // to the terminal.
604  checkFile(filename);
605 
606  m_file = H5Fopen(filename.c_str(), H5F_ACC_RDONLY, H5P_DEFAULT);
607 
608  if (m_file < 0)
609  throw NoSuchFileException(filename);
610 
611  int fileVersion[3];
612  try {
613  if (!readAttribute(m_file, k_versionAttrName, 3, fileVersion[0])) {
614  //Msg::print(Msg::SevWarning, "Missing version_number attribute");
615  } else {
616  if (!isSupportedFileVersion(fileVersion, k_minFileVersion)) {
617  stringstream versionStr;
618  versionStr << fileVersion[0] << "."
619  << fileVersion[1] << "."
620  << fileVersion[2];
621  throw UnsupportedVersionException(versionStr.str());
622  }
623  }
624  }
625  catch (MissingAttributeException &) {
626  //Msg::print(Msg::SevWarning, "Missing version_number attribute");
627  }
628 
629  try {
630  if (H5Lexists(m_file, "field3d_global_metadata", H5P_DEFAULT)) {
631  // read the metadata
632  H5ScopedGopen metadataGroup(m_file, "field3d_global_metadata");
633  if (metadataGroup.id() > 0) {
634  readMetadata(metadataGroup.id());
635  }
636  }
637  }
638  catch (...) {
640  "Unknown error when reading file metadata ");
641  //throw BadFileHierarchyException(filename);
642  }
643 
644  try {
645  if (!readPartitionAndLayerInfo()) {
646  success = false;
647  }
648  }
649  catch (MissingGroupException &e) {
650  Msg::print(Msg::SevWarning, "Missing group: " + string(e.what()));
651  throw BadFileHierarchyException(filename);
652  }
653  catch (ReadMappingException &e) {
654  Msg::print(Msg::SevWarning, "Couldn't read mapping for partition: "
655  + string(e.what()));
656  throw BadFileHierarchyException(filename);
657  }
658  catch (Exception &e) {
659  Msg::print(Msg::SevWarning, "Unknown error when reading file hierarchy: "
660  + string(e.what()));
661  throw BadFileHierarchyException(filename);
662  }
663  catch (...) {
665  "Unknown error when reading file hierarchy. ");
666  throw BadFileHierarchyException(filename);
667  }
668 
669  }
670  catch (NoSuchFileException &e) {
671  Msg::print(Msg::SevWarning, "Couldn't open file: "
672  + string(e.what()) );
673  success = false;
674  }
675  catch (MissingAttributeException &e) {
677  "In file: " + filename + " - "
678  + string(e.what()) );
679  success = false;
680  }
681  catch (UnsupportedVersionException &e) {
683  "In file: " + filename + " - File version can not be read: "
684  + string(e.what()));
685  success = false;
686  }
687  catch (BadFileHierarchyException &) {
689  "In file: " + filename + " - Bad file hierarchy. ");
690  success = false;
691  }
692  catch (...) {
694  "In file: " + filename + " Unknown exception ");
695  success = false;
696  }
697 
698  if (!success)
699  close();
700 
701  return success;
702 }

References Field3DFileHDF5Base::clear(), Field3DFileHDF5Base::close(), g_hdf5Mutex, Hdf5Util::H5Base::id(), Field3DFileHDF5Base::m_file, m_filename, Msg::print(), Hdf5Util::readAttribute(), readMetadata(), readPartitionAndLayerInfo(), Msg::SevWarning, and Exc::Exception::what().

◆ parsePartition()

herr_t Field3DInputFileHDF5::parsePartition ( hid_t  loc_id,
const std::string  partitionName 
)

Gets called from parsePartitions. Not intended for any other use.

Definition at line 800 of file Field3DFileHDF5.cpp.

802 {
803  // Add the partition ---
804 
805  m_partitionNames.push_back(string(itemName));
806  return 0;
807 }

References Field3DFileHDF5Base::m_partitionNames.

Referenced by InputFileHDF5::parsePartitions().

◆ parseLayer()

herr_t Field3DInputFileHDF5::parseLayer ( hid_t  layerGroup,
const std::string &  partitionName,
const std::string &  layerName 
)

Gets called from parsePartitions. Not intended for any other use.

Note
Don't throw exceptions into the hdf5 lib.
Todo:
Set some sort of flag if we fail during this call. We can't throw exceptions inside hdf5.

Definition at line 814 of file Field3DFileHDF5.cpp.

817 {
818  int components;
819  if (!readAttribute(layerGroup, string("components"), 1, components)) {
820  Msg::print(Msg::SevWarning, "Couldn't read components attribute for layer "
821  + partitionName + "/" + layerName);
822  return 0;
823  }
824 
825  LayerInfo linfo(partitionName,layerName,components);
826 
827  m_layerInfo.push_back(linfo);
828 
829  return 0;
830 }

References Field3DFileHDF5Base::m_layerInfo, Msg::print(), Hdf5Util::readAttribute(), and Msg::SevWarning.

Referenced by InputFileHDF5::parseLayers().

◆ readGroupMembership()

bool Field3DInputFileHDF5::readGroupMembership ( GroupMembershipMap gpMembershipMap)

Read the group membership for the partitions.

Definition at line 1056 of file Field3DFileHDF5.cpp.

1058 {
1059  GlobalLock lock(g_hdf5Mutex);
1060 
1061  if (!H5Lexists(m_file, "field3d_group_membership", H5P_DEFAULT)) {
1062  return false;
1063  }
1064 
1065  H5ScopedGopen memberGroup(m_file, "field3d_group_membership");
1066  if (memberGroup < 0) {
1067  return false;
1068  }
1069 
1070  typedef boost::tokenizer<boost::char_separator<char> > Tok;
1071 
1072  hsize_t num_attrs = H5Aget_num_attrs(memberGroup);
1073  if (num_attrs > 0) {
1074 
1075  for (hsize_t idx=0; idx < num_attrs ; ++idx) {
1076  H5ScopedAopenIdx attrIdx(memberGroup, idx);
1077  size_t len = H5Aget_name(attrIdx.id(), 0, NULL);
1078  if (len>0) {
1079  char *name = new char[len+1];
1080  if (H5Aget_name(attrIdx.id(), len+1, name) > 0) {
1081 
1082  if (string(name) == "is_field3d_group_membership")
1083  continue;
1084 
1085  H5ScopedAopen attr(memberGroup, name, H5P_DEFAULT);
1086  H5ScopedAget_space attrSpace(attr);
1087  H5ScopedAget_type attrType(attr);
1088  H5T_class_t typeClass = H5Tget_class(attrType);
1089 
1090  if (typeClass == H5T_STRING) {
1091  string value;
1092  if (!readAttribute(memberGroup, name, value)) {
1094  "Failed to read group membership data "
1095  + string(name));
1096  continue;
1097  }
1098 
1099  {
1100  boost::char_separator<char> sep(" :");
1101  Tok tok(value, sep);
1102  string new_value;
1103  for(Tok::iterator beg=tok.begin(); beg!=tok.end();){
1104 
1105  string fieldgroup = *beg; ++beg;
1106  fieldgroup = removeUniqueId(fieldgroup) + ":" + *beg; ++beg;
1107  new_value += fieldgroup + " ";
1108  }
1109 
1110  m_groupMembership[name] = value;
1111  gpMembershipMap[name] = new_value;
1112  }
1113  }
1114  }
1115  }
1116  }
1117  }
1118 
1119  return true;
1120 }

References g_hdf5Mutex, Hdf5Util::H5Base::id(), Field3DFileHDF5Base::m_file, Field3DFileHDF5Base::m_groupMembership, Msg::print(), Hdf5Util::readAttribute(), Field3DFileHDF5Base::removeUniqueId(), and Msg::SevWarning.

◆ readScalarLayer()

template<class Data_T >
Field< Data_T >::Ptr Field3DInputFileHDF5::readScalarLayer ( const std::string &  intPartitionName,
const std::string &  layerName 
) const
private

Retrieves a single layer given its and its parent partition's name. Maintains the on-disk data types.

Definition at line 1359 of file Field3DFileHDF5.h.

1361 {
1362  return readLayer<Data_T>(intPartitionName, layerName, false);
1363 }

References Field3DFileHDF5Base::intPartitionName().

◆ readVectorLayer()

template<class Data_T >
Field< FIELD3D_VEC3_T< Data_T > >::Ptr Field3DInputFileHDF5::readVectorLayer ( const std::string &  intPartitionName,
const std::string &  layerName 
) const
private

Retrieves a single layer given its and its parent partition's name. Maintains the on-disk data types.

Definition at line 1369 of file Field3DFileHDF5.h.

1371 {
1372  return readLayer<FIELD3D_VEC3_T<Data_T> >(intPartitionName, layerName, true);
1373 }

References Field3DFileHDF5Base::intPartitionName().

◆ readLayer()

template<class Data_T >
Field< Data_T >::Ptr Field3DInputFileHDF5::readLayer ( const std::string &  intPartitionName,
const std::string &  layerName,
bool  isVectorLayer 
) const
private

This call does the actual reading of a layer. Notice that it expects a unique -internal- partition name.

Definition at line 1052 of file Field3DFileHDF5.h.

1055 {
1056  using namespace boost;
1057  using namespace std;
1058  using namespace Hdf5Util;
1059 
1060  typedef typename Field<Data_T>::Ptr FieldPtr;
1061 
1062  // Instantiate a null pointer for easier code reading
1063  FieldPtr nullPtr;
1064 
1065  GlobalLock lock(g_hdf5Mutex);
1066 
1067  // Find the partition
1069  if (!part) {
1070  Msg::print(Msg::SevWarning, "Couldn't find partition: " + intPartitionName);
1071  return nullPtr;
1072  }
1073 
1074  // Find the layer in the partition
1075  const FileHDF5::Layer *l;
1076  if (isVectorLayer)
1077  l = part->vectorLayer(layerName);
1078  else
1079  l = part->scalarLayer(layerName);
1080  if (!l) {
1081  Msg::print(Msg::SevWarning, "Couldn't find layer: " + layerName );
1082  return nullPtr;
1083  }
1084 
1085  // Open the layer group
1086  string layerPath = l->parent + "/" + l->name;
1087  H5ScopedGopen layerGroup(m_file, layerPath.c_str());
1088 
1089  if (layerGroup.id() < 0) {
1090  Msg::print(Msg::SevWarning, "Couldn't find layer group " + layerName
1091  + " in .f3d file ");
1092  return nullPtr;
1093  }
1094 
1095  // Get the class name
1096  string className;
1097  if (!readAttribute(layerGroup.id(), "class_name", className)) {
1098  Msg::print(Msg::SevWarning, "Couldn't find class_name attrib in layer " +
1099  layerName);
1100  return nullPtr;
1101  }
1102 
1103  // Check the cache
1104 
1106  FieldPtr cachedField = cache.getCachedField(m_filename, layerPath);
1107 
1108  if (cachedField) {
1109  return cachedField;
1110  }
1111 
1112  // Construct the field and load the data
1113 
1114  // Unlock the g_hdf5Mutex while calling readField() so that other threads
1115  // have a chance to pre-empt loading in between (prevents deadlocks with
1116  // sparse reader)
1117  lock.unlock();
1118 
1119  typename Field<Data_T>::Ptr field;
1120  field = readField<Data_T>(className, layerGroup.id(), m_filename, layerPath);
1121 
1122  if (!field) {
1123 #if 0 // This isn't really an error
1124  Msg::print(Msg::SevWarning, "Couldn't read the layer data of layer: "
1125  + layerName);
1126 #endif
1127  return nullPtr;
1128  }
1129 
1130  // Now we need to use Hdf5 again, so re-aquire the lock.
1131  lock.lock();
1132 
1133  // read the metadata
1134  string metadataPath = layerPath + "/metadata";
1135  H5ScopedGopen metadataGroup(m_file, metadataPath.c_str());
1136  if (metadataGroup.id() > 0) {
1137  readMetadata(metadataGroup.id(), field);
1138  }
1139 
1140  // Set the name of the field so it's possible to re-create the file
1142  field->attribute = layerName;
1143  field->setMapping(part->mapping);
1144 
1145  // Cache the field for future use
1146  if (field) {
1147  cache.cacheField(field, m_filename, layerPath);
1148  }
1149 
1150  return field;
1151 }

References FieldBase::attribute, FieldCache< Data_T >::cacheField(), g_hdf5Mutex, FieldCache< Data_T >::getCachedField(), Hdf5Util::H5Base::id(), Field3DFileHDF5Base::intPartitionName(), Field3DFileHDF5Base::m_file, m_filename, FileHDF5::Layer::name, FieldBase::name, FileHDF5::Layer::parent, Field3DFileHDF5Base::partition(), Msg::print(), Hdf5Util::readAttribute(), readMetadata(), Field3DFileHDF5Base::removeUniqueId(), FieldRes::setMapping(), Msg::SevWarning, and FieldCache< Data_T >::singleton().

◆ readPartitionAndLayerInfo()

bool Field3DInputFileHDF5::readPartitionAndLayerInfo ( )
private

Sets up all the partitions and layers, but does not load any data.

Definition at line 706 of file Field3DFileHDF5.cpp.

707 {
708  using namespace InputFileHDF5;
709 
710  GlobalLock lock(g_hdf5Mutex);
711 
712  // First, find the partitions ---
713 
714  herr_t status;
715  status = H5Literate(m_file, H5_INDEX_NAME, H5_ITER_NATIVE, NULL,
716  &parsePartitions, this);
717 
718  // Get the partition names to store
719  m_partitions.clear();
720 
721  for (size_t i=0; i < m_partitionNames.size(); i++) {
722  Partition::Ptr part(new Partition);
723  part->name = m_partitionNames[i];
724  m_partitions.push_back(part);
725  }
726 
727  // For each partition, find its mapping ---
728 
729  for (PartitionList::iterator i = m_partitions.begin();
730  i != m_partitions.end(); ++i) {
731 
732  // Open the partition
733  H5ScopedGopen partitionGroup(m_file, (**i).name);
734 
735  string mappingPath = "/" + (**i).name + "/" + k_mappingStr;
736 
737  // Open up the mapping group
738  H5ScopedGopen mappingGroup(m_file, mappingPath);
739  if (mappingGroup.id() < 0)
740  throw MissingGroupException((**i).name + "/" + k_mappingStr);
741 
742  // Try to build a mapping from it
743  FieldMapping::Ptr mapping;
744 
745  mapping = readFieldMapping(mappingGroup.id());
746  if (!mapping) {
747  Msg::print(Msg::SevWarning, "Got a null pointer when reading mapping");
748  throw ReadMappingException((**i).name);
749  }
750 
751  // Attach the mapping to the partition
752  (**i).mapping = mapping;
753 
754  }
755 
756  // ... And then find its layers ---
757 
758  for (PartitionList::const_iterator i = m_partitions.begin();
759  i != m_partitions.end(); ++i) {
760 
761  // Open the partition
762  H5ScopedGopen partitionGroup(m_file, (**i).name);
763 
764  // Set up the info struct for the callback
765  ParseLayersInfo info;
766  info.file = this;
767  info.partitionName = (**i).name;
768 
769  m_layerInfo.clear();
770 
771  status = H5Literate(partitionGroup.id(), H5_INDEX_NAME, H5_ITER_NATIVE,
772  NULL, &parseLayers, &info);
773 
774  //set the layer information on the partitions here
775 
776  for (std::vector<LayerInfo>::iterator i = m_layerInfo.begin();
777  i != m_layerInfo.end(); i++) {
778 
779  std::string parent = i->parentName;
780 
781  Partition::Ptr part = partition(parent);
782 
783  Layer layer;
784  layer.name = i->name;
785  layer.parent = i->parentName;
786  if (i->components == 1) {
787  part->addScalarLayer(layer);
788  } else if (i->components == 3) {
789  part->addVectorLayer(layer);
790  }
791  }
792 
793  }
794 
795  return true;
796 }

References InputFileHDF5::ParseLayersInfo::file, g_hdf5Mutex, Hdf5Util::H5Base::id(), Field3DFileHDF5Base::m_file, Field3DFileHDF5Base::m_layerInfo, Field3DFileHDF5Base::m_partitionNames, Field3DFileHDF5Base::m_partitions, FileHDF5::Layer::name, FileHDF5::Layer::parent, InputFileHDF5::parseLayers(), InputFileHDF5::parsePartitions(), Field3DFileHDF5Base::partition(), InputFileHDF5::ParseLayersInfo::partitionName, Msg::print(), readFieldMapping(), and Msg::SevWarning.

Referenced by open().

◆ readMetadata() [1/2]

bool Field3DInputFileHDF5::readMetadata ( hid_t  metadata_id,
FieldBase::Ptr  field 
) const
private

Read metadata for this layer.

Todo:
Replace char* with std::string

Definition at line 836 of file Field3DFileHDF5.cpp.

838 {
839  GlobalLock lock(g_hdf5Mutex);
840 
841  hsize_t num_attrs = H5Aget_num_attrs(metadata_id);
842 
843  if (num_attrs > 0) {
844  for (hsize_t idx=0; idx < num_attrs ; ++idx) {
845  H5ScopedAopenIdx attrIdx(metadata_id, idx);
846  size_t len = H5Aget_name(attrIdx.id(), 0, NULL);
847  if (len > 0) {
848  char *name = new char[len+1];
849  if (H5Aget_name(attrIdx.id(), len+1, name) > 0) {
850  H5ScopedAopen attr(metadata_id, name, H5P_DEFAULT);
851  H5ScopedAget_space attrSpace(attr);
852  H5ScopedAget_type attrType(attr);
853  H5T_class_t typeClass = H5Tget_class(attrType);
854 
855  if (typeClass == H5T_STRING) {
856  string value;
857  if (!readAttribute(metadata_id, name, value)) {
859  "Failed to read metadata " + string(name));
860  if (name) {
861  delete[] name;
862  }
863  continue;
864  }
865  field->metadata().setStrMetadata(name, value);
866 
867  }
868  else {
869 
870  if (H5Sget_simple_extent_ndims(attrSpace) != 1) {
871  Msg::print(Msg::SevWarning, "Bad attribute rank for attribute "
872  + string(name));
873  if (name) {
874  delete[] name;
875  }
876  continue;
877  }
878 
879  hsize_t dims[1];
880  H5Sget_simple_extent_dims(attrSpace, dims, NULL);
881 
882  if (typeClass == H5T_INTEGER) {
883  if (dims[0] == 1){
884  int value;
885  if (!readAttribute(metadata_id, name, dims[0], value))
886  Msg::print(Msg::SevWarning, "Failed to read metadata "
887  + string(name));
888  field->metadata().setIntMetadata(name, value);
889  }
890  else if (dims[0] == 3){
891  V3i value;
892  if (!readAttribute(metadata_id, name, dims[0], value.x))
893  Msg::print(Msg::SevWarning, "Failed to read metadata " +
894  string(name) );
895  field->metadata().setVecIntMetadata(name, value);
896  }
897  else {
899  "Attribute of size " +
900  boost::lexical_cast<std::string>(dims[0])
901  + " is not valid for metadata");
902  }
903  }
904  else if (typeClass == H5T_FLOAT) {
905  if (dims[0] == 1){
906  float value;
907  if (!readAttribute(metadata_id, name, dims[0], value))
908  Msg::print(Msg::SevWarning, "Failed to read metadata " +
909  string(name) );
910 
911  field->metadata().setFloatMetadata(name, value);
912  }
913  else if (dims[0] == 3){
914  V3f value;
915  if (!readAttribute(metadata_id, name, dims[0], value.x))
916  Msg::print(Msg::SevWarning, "Failed to read metadata "+
917  string(name) );
918  field->metadata().setVecFloatMetadata(name, value);
919  }
920  else {
921  Msg::print(Msg::SevWarning, "Attribute of size " +
922  boost::lexical_cast<std::string>(dims[0]) +
923  " is not valid for metadata");
924  }
925  }
926  else {
927  Msg::print(Msg::SevWarning, "Attribute '" + string(name) +
928  + "' has unsupported data type for metadata");
929 
930  }
931  }
932  }
933  if (name) {
934  delete[] name;
935  }
936  }
937  }
938  }
939 
940  return true;
941 }

References g_hdf5Mutex, Hdf5Util::H5Base::id(), Msg::print(), Hdf5Util::readAttribute(), and Msg::SevWarning.

Referenced by open(), readLayer(), and readProxyLayer().

◆ readMetadata() [2/2]

bool Field3DInputFileHDF5::readMetadata ( hid_t  metadata_id)
private

Read global metadata for this file.

Todo:
Replace char* with std::string

Definition at line 947 of file Field3DFileHDF5.cpp.

948 {
949  GlobalLock lock(g_hdf5Mutex);
950 
951  hsize_t num_attrs = H5Aget_num_attrs(metadata_id);
952 
953  if (num_attrs > 0) {
954  for (hsize_t idx=0; idx < num_attrs ; ++idx) {
955  H5ScopedAopenIdx attrIdx(metadata_id, idx);
956  size_t len = H5Aget_name(attrIdx.id(), 0, NULL);
957  if (len > 0) {
958  char *name = new char[len+1];
959  if (H5Aget_name(attrIdx.id(), len+1, name) > 0) {
960  H5ScopedAopen attr(metadata_id, name, H5P_DEFAULT);
961  H5ScopedAget_space attrSpace(attr);
962  H5ScopedAget_type attrType(attr);
963  H5T_class_t typeClass = H5Tget_class(attrType);
964 
965  if (typeClass == H5T_STRING) {
966  string value;
967  if (!readAttribute(metadata_id, name, value)) {
969  "Failed to read metadata " + string(name));
970  if (name) {
971  delete[] name;
972  }
973  continue;
974  }
975  metadata().setStrMetadata(name, value);
976 
977  }
978  else {
979 
980  if (H5Sget_simple_extent_ndims(attrSpace) != 1) {
981  Msg::print(Msg::SevWarning, "Bad attribute rank for attribute "
982  + string(name));
983  if (name) {
984  delete[] name;
985  }
986  continue;
987  }
988 
989  hsize_t dims[1];
990  H5Sget_simple_extent_dims(attrSpace, dims, NULL);
991 
992  if (typeClass == H5T_INTEGER) {
993  if (dims[0] == 1){
994  int value;
995  if (!readAttribute(metadata_id, name, dims[0], value))
996  Msg::print(Msg::SevWarning, "Failed to read metadata "
997  + string(name));
998  metadata().setIntMetadata(name, value);
999  }
1000  else if (dims[0] == 3){
1001  V3i value;
1002  if (!readAttribute(metadata_id, name, dims[0], value.x))
1003  Msg::print(Msg::SevWarning, "Failed to read metadata " +
1004  string(name) );
1005  metadata().setVecIntMetadata(name, value);
1006  }
1007  else {
1009  "Attribute of size " +
1010  boost::lexical_cast<std::string>(dims[0])
1011  + " is not valid for metadata");
1012  }
1013  }
1014  else if (typeClass == H5T_FLOAT) {
1015  if (dims[0] == 1){
1016  float value;
1017  if (!readAttribute(metadata_id, name, dims[0], value))
1018  Msg::print(Msg::SevWarning, "Failed to read metadata " +
1019  string(name) );
1020 
1021  metadata().setFloatMetadata(name, value);
1022  }
1023  else if (dims[0] == 3){
1024  V3f value;
1025  if (!readAttribute(metadata_id, name, dims[0], value.x))
1026  Msg::print(Msg::SevWarning, "Failed to read metadata "+
1027  string(name) );
1028  metadata().setVecFloatMetadata(name, value);
1029  }
1030  else {
1031  Msg::print(Msg::SevWarning, "Attribute of size " +
1032  boost::lexical_cast<std::string>(dims[0]) +
1033  " is not valid for metadata");
1034  }
1035  }
1036  else {
1037  Msg::print(Msg::SevWarning, "Attribute '" + string(name) +
1038  + "' has unsupported data type for metadata");
1039 
1040  }
1041  }
1042  }
1043  if (name) {
1044  delete[] name;
1045  }
1046  }
1047  }
1048  }
1049 
1050  return true;
1051 }

References g_hdf5Mutex, Hdf5Util::H5Base::id(), Field3DFileHDF5Base::metadata(), Msg::print(), Hdf5Util::readAttribute(), FieldMetadata::setFloatMetadata(), FieldMetadata::setIntMetadata(), FieldMetadata::setStrMetadata(), FieldMetadata::setVecFloatMetadata(), FieldMetadata::setVecIntMetadata(), and Msg::SevWarning.

Friends And Related Function Documentation

◆ Field3DInputFile

friend class Field3DInputFile
friend

Definition at line 441 of file Field3DFileHDF5.h.

◆ Field3DOutputFile

friend class Field3DOutputFile
friend

Definition at line 442 of file Field3DFileHDF5.h.

Member Data Documentation

◆ m_filename

std::string Field3DInputFileHDF5::m_filename
private

Filename, only to be set by open().

Definition at line 736 of file Field3DFileHDF5.h.

Referenced by open(), and readLayer().


The documentation for this class was generated from the following files:
Field3DFileHDF5Base::m_groupMembership
GroupMembershipMap m_groupMembership
Keeps track of group membership for each layer of partition name. The key is the "group" and the valu...
Definition: Field3DFileHDF5.h:403
Field3DFileHDF5Base::m_partitionNames
std::vector< std::string > m_partitionNames
This stores partition names.
Definition: Field3DFileHDF5.h:394
V3i
Imath::V3i V3i
Definition: SpiMathLib.h:71
Exc::Exception::what
virtual const char * what() const
Definition: Exception.h:90
Field3DFileHDF5Base::getScalarLayerNames
void getScalarLayerNames(std::vector< std::string > &names, const std::string &partitionName) const
Gets the names of all the scalar layers in a given partition.
Definition: Field3DFileHDF5.cpp:403
Msg::SevWarning
@ SevWarning
Definition: Log.h:68
Field3DFileHDF5Base::close
bool close()
Closes the file. No need to call this unless you specifically want to close the file early....
Definition: Field3DFileHDF5.cpp:496
FieldBase::metadata
FieldMetadata & metadata()
accessor to the m_metadata class
Definition: Field.h:155
EmptyField::Vec
std::vector< Ptr > Vec
Definition: EmptyField.h:94
Field::Ptr
boost::intrusive_ptr< Field > Ptr
Definition: Field.h:395
InputFileHDF5::ParseLayersInfo::file
Field3DInputFileHDF5 * file
Definition: Field3DFileHDF5.h:886
Field3DFileHDF5Base::clear
void clear()
Clear the data structures and close the file.
Definition: Field3DFileHDF5.cpp:487
FileHDF5::Layer::name
std::string name
The name of the layer (always available)
Definition: Field3DFileHDF5.h:124
Field3DFileHDF5Base::partition
FileHDF5::Partition::Ptr partition(const std::string &partitionName)
Returns a pointer to the given partition.
Field3DInputFileHDF5::readPartitionAndLayerInfo
bool readPartitionAndLayerInfo()
Sets up all the partitions and layers, but does not load any data.
Definition: Field3DFileHDF5.cpp:706
InputFileHDF5::parsePartitions
FIELD3D_API herr_t parsePartitions(hid_t loc_id, const char *partitionName, const H5L_info_t *linfo, void *opdata)
Gets called from readPartitionAndLayerInfo to check each group found under the root of the file....
Definition: Field3DFileHDF5.cpp:1130
InputFileHDF5::ParseLayersInfo
struct used to pass the class and partition info back to the parseLayers() callback
Definition: Field3DFileHDF5.h:885
Field3DFileHDF5Base::removeUniqueId
std::string removeUniqueId(const std::string &partitionName) const
Strips any unique identifiers from the partition name and returns the original name.
Definition: Field3DFileHDF5.cpp:373
Field3DFileHDF5Base::m_file
hid_t m_file
The hdf5 id of the current file. Will be -1 if no file is open.
Definition: Field3DFileHDF5.h:390
FieldCache::singleton
static FieldCache & singleton()
Returns a reference to the FieldCache singleton.
Definition: FieldCache.h:135
FieldMapping::Ptr
boost::intrusive_ptr< FieldMapping > Ptr
Definition: FieldMapping.h:92
EmptyField::Ptr
boost::intrusive_ptr< EmptyField > Ptr
Definition: EmptyField.h:93
V3f
Imath::V3f V3f
Definition: SpiMathLib.h:73
FileHDF5::Layer
Definition: Field3DFileHDF5.h:121
FieldBase::name
std::string name
Optional name of the field.
Definition: Field.h:171
InputFileHDF5::parseLayers
FIELD3D_API herr_t parseLayers(hid_t loc_id, const char *partitionName, const H5L_info_t *linfo, void *opdata)
Gets called from readPartitionAndLayerInfo to check each group found under the root of the file....
Definition: Field3DFileHDF5.cpp:1170
Field3DFileHDF5Base::getIntVectorLayerNames
void getIntVectorLayerNames(std::vector< std::string > &names, const std::string &intPartitionName) const
Gets the names of all the vector layers in a given partition, but assumes that partition name is the ...
Definition: Field3DFileHDF5.cpp:470
FieldRes::setMapping
void setMapping(FieldMapping::Ptr mapping)
Sets the field's mapping.
Definition: Field.h:347
Hdf5Util::H5ScopedGopen
Scoped object - opens a group on creation and closes it on destruction.
Definition: Hdf5Util.h:195
FileHDF5::Layer::parent
std::string parent
The name of the parent partition. We need this in order to open its group.
Definition: Field3DFileHDF5.h:127
Field3DFileHDF5Base::getIntScalarLayerNames
void getIntScalarLayerNames(std::vector< std::string > &names, const std::string &intPartitionName) const
Gets the names of all the scalar layers in a given partition, but assumes that partition name is the ...
Definition: Field3DFileHDF5.cpp:452
Exc::Exception
Definition: Exception.h:74
FieldMetadata::setStrMetadata
void setStrMetadata(const std::string &name, const std::string &val)
Set the a string value for the given metadata name.
Definition: FieldMetadata.cpp:102
FieldMetadata::setIntMetadata
void setIntMetadata(const std::string &name, const int val)
Set the a int value for the given metadata name.
Definition: FieldMetadata.cpp:91
Field3DFileHDF5Base::m_partitions
PartitionList m_partitions
Vector of partitions.
Definition: Field3DFileHDF5.h:392
FieldBase::attribute
std::string attribute
Optional name of the attribute the field represents.
Definition: Field.h:173
Field::Vec
std::vector< Ptr > Vec
This is a convenience typedef for the list that Field3DInputFile::readScalarLayers() and Field3DInput...
Definition: Field.h:403
FieldMetadata::setFloatMetadata
void setFloatMetadata(const std::string &name, const float val)
Set the a float value for the given metadata name.
Definition: FieldMetadata.cpp:69
Field3DInputFileHDF5::readMetadata
bool readMetadata(hid_t metadata_id, FieldBase::Ptr field) const
Read metadata for this layer.
Definition: Field3DFileHDF5.cpp:837
Hdf5Util::H5ScopedAget_space
Scoped object - opens an attribute data space on creation and closes it on destruction.
Definition: Hdf5Util.h:288
g_hdf5Mutex
FIELD3D_NAMESPACE_OPEN FIELD3D_API boost::recursive_mutex g_hdf5Mutex
Definition: Hdf5Util.cpp:67
Hdf5Util::H5ScopedAopenIdx
Scoped object - Opens attribute by index and closes it on destruction.
Definition: Hdf5Util.h:143
Field3DFileHDF5Base::metadata
FieldMetadata & metadata()
accessor to the m_metadata class
Definition: Field3DFileHDF5.h:321
EmptyField
This subclass of Field does not store any data.
Definition: EmptyField.h:88
InputFileHDF5::ParseLayersInfo::partitionName
std::string partitionName
Definition: Field3DFileHDF5.h:887
FieldMetadata::setVecFloatMetadata
void setVecFloatMetadata(const std::string &name, const V3f &val)
Set the a V3f value for the given metadata name.
Definition: FieldMetadata.cpp:58
GlobalLock
boost::recursive_mutex::scoped_lock GlobalLock
Definition: Hdf5Util.h:78
Hdf5Util::readAttribute
FIELD3D_API bool readAttribute(hid_t location, const std::string &attrName, std::string &value)
Reads a string attribute.
Field3DFileHDF5Base::intPartitionName
std::string intPartitionName(const std::string &partitionName, const std::string &layerName, FieldRes::Ptr field)
Returns a unique partition name given the requested name. This ensures that partitions with matching ...
Definition: Field3DFileHDF5.cpp:316
Hdf5Util
Contains utility functions and classes for Hdf5 files.
Definition: Hdf5Util.h:86
Field3DFileHDF5Base::getIntPartitionNames
void getIntPartitionNames(std::vector< std::string > &names) const
Gets the names of all the -internal- partitions in the file.
Definition: Field3DFileHDF5.cpp:439
Field3DInputFileHDF5::m_filename
std::string m_filename
Filename, only to be set by open().
Definition: Field3DFileHDF5.h:736
FieldCache::getCachedField
FieldPtr getCachedField(const std::string &filename, const std::string &layerPath)
Checks the cache for a previously loaded field.
Definition: FieldCache.h:148
readFieldMapping
FieldMapping::Ptr readFieldMapping(hid_t mappingGroup)
This function creates a FieldMappingIO instance based on className read from mappingGroup location wh...
Definition: Field3DFileHDF5.cpp:1613
Hdf5Util::H5ScopedAopen
Scoped object - Opens attribute by name and closes it on destruction.
Definition: Hdf5Util.h:114
Field3DFileHDF5Base::getPartitionNames
void getPartitionNames(std::vector< std::string > &names) const
Gets the names of all the partitions in the file.
Definition: Field3DFileHDF5.cpp:386
InputFileHDF5
Namespace for file input specifics.
Definition: Field3DFileHDF5.h:879
Box3i
Imath::Box3i Box3i
Definition: SpiMathLib.h:77
FileHDF5::Partition
Definition: Field3DFileHDF5.h:146
FieldCache::cacheField
void cacheField(FieldPtr field, const std::string &filename, const std::string &layerPath)
Adds the given field to the cache.
Definition: FieldCache.h:169
Msg::print
FIELD3D_API void print(Severity severity, const std::string &message)
Sends the string to the assigned output, prefixing the message with the severity.
Definition: Log.cpp:70
Field3DFileHDF5Base::m_layerInfo
std::vector< LayerInfo > m_layerInfo
This stores layer info.
Definition: Field3DFileHDF5.h:387
Field
Definition: Field.h:390
Field3DFileHDF5Base::getVectorLayerNames
void getVectorLayerNames(std::vector< std::string > &names, const std::string &partitionName) const
Gets the names of all the vector layers in a given partition.
Definition: Field3DFileHDF5.cpp:421
Hdf5Util::H5ScopedAget_type
Scoped object - opens an attribute data type on creation and closes it on destruction.
Definition: Hdf5Util.h:311
FieldCache
Definition: FieldCache.h:81
FieldMetadata::setVecIntMetadata
void setVecIntMetadata(const std::string &name, const V3i &val)
Set the a V3i value for the given metadata name.
Definition: FieldMetadata.cpp:80
FileHDF5::Partition::Ptr
boost::intrusive_ptr< Partition > Ptr
Definition: Field3DFileHDF5.h:152