hikari.dataframes ================= .. py:module:: hikari.dataframes .. autoapi-nested-parse:: This module contains all dataframes utilised in hikari. A dataframe is a low-level object, which stores and manipulates certain crystallographic information. At the moment, the following dataframes are implemented: - **hkl** - for single crystal reflection data - **cif** - for crystallographic open format data (partially) - **res** - for shelx crystal structure data (partially) Please mind that the `hkl` frame, HklFrame is the most developed. Other frames are in an early stage of development. Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/hikari/dataframes/base/index /autoapi/hikari/dataframes/cif/index /autoapi/hikari/dataframes/hkl/index /autoapi/hikari/dataframes/lst/index /autoapi/hikari/dataframes/res/index /autoapi/hikari/dataframes/ubase/index Classes ------- .. autoapisummary:: hikari.dataframes.CifFrame hikari.dataframes.CifBlock hikari.dataframes.BaseFrame hikari.dataframes.UBaseFrame hikari.dataframes.HklFrame hikari.dataframes.ResFrame hikari.dataframes.LstFrame Package Contents ---------------- .. py:class:: CifFrame(dict=None, /, **kwargs) Bases: :py:obj:`collections.UserDict` A master object which manages cif files. It utilises other `Cif*` classes to manage multiple :class:`CifBlock`s with crystallographic information. As a subclass of an `UserDict`, in python3.7+ it is ordered by design. Individual Cif blocks and items within them can be accessed or assigned using a single- or nested- dict-like syntax. Similarly to other `Frame`s, `CifFrame` is designed to work in-place, meaning it should be first created, and only then accessed using methods such as :func:`read` or :func:`write`, but not chain assignments. Unlike dict, CifBlock always initiates empty and does not accept any parameters at creation. .. py:method:: read(path) Read the contents of .cif file specified by the `path` parameter. Store each found block as a {block_name: CifBlock} pair. :param path: Absolute or relative path to the .cif file. .. py:method:: write(path) Write the contents of `CifFrame` to the .cif file specified by the `path` parameter. :param path: Absolute or relative path to the .cif file. .. py:class:: CifBlock(dict=None, /, **kwargs) Bases: :py:obj:`collections.UserDict` CifBlock object handles all data inside an individual block of Cif file. As a subclass of an `UserDict`, in python3.7+ it is ordered by design. Individual Cif items can be accessed or assigned using a dict-like syntax. .. py:method:: get_as_type(key, typ, default = None) Get value of `self[key]` converted to `typ`. If value is a list, convert its contents element-wise. :param key: key associated with accessed element :param typ: type/function applied to a value or its every element :param default: if given, return it on KeyError :return: value of `self[key]` or `default` converted to `typ` .. py:method:: read(path, block) Read the contents of .cif file specified by the `path` parameter, but access and store only the `block` data block in self. :param path: Absolute or relative path to the .cif file. :param block: Name of the cif data block to be accessed .. py:method:: write(path) Write the contents of `CifBlock` to the .cif file specified by the `path` parameter, using 'hikari' as block name. :param path: Absolute or relative path to the .cif file. .. py:class:: BaseFrame This class stores and manipulates basic information present in majority of crystallographic information files such as unit cell parameters stored in scalars and vectors. BaseFrame utilises the following notation for stored attributes: - The name begins from a unit cell property we are interested in: - "a", "b", "c" describe unit cell lengths/vectors *a*, *b*, *c*, - "al", "be", "ga" describe unit cell angles *alpha*, *beta*, *gamma*, - "v" describes unit cell volume, - "x", "y", "z" describe directions - normalised unit cell vectors. - "A", "G" describe stacked vector and metric matrix, respectively. - The unit cell parameter symbol is then followed by an underscore "_". - The name ends with a single letter denoting type of space and variable: - "d" (from Direct) denotes direct space scalars/matrices, - "r" (from Reciprocal) denotes reciprocal space scalars/matrices, - "v" (from Vector) denotes direct space vectors, - 'w" (similar to "v") denotes reciprocal space vectors. The values can be accessed by referencing a given attribute in the object, for example :class:`BaseFrame`. :attr:`a_d` stores information about the lattice constant *a* in direct space as a floating point, but :class:`BaseFrame`. :attr:`a_v` is a direct space vector. Available attributes have been once again presented in a table below: +----------+------------+------------------+------------------+------------+ | | Available | in direct | in reciprocal |Unit (^-1 in| | | constants | space | space |reciprocal) | +==========+============+==================+==================+============+ | Scalars | a, b, c | a_d, b_d, c_d | a_r, b_r, c_r | Angstrom | | +------------+------------------+------------------+------------+ | | al, be, ga | al_d, be_d, ga_d | al_r, be_r, ga_r | Radian | | +------------+------------------+------------------+------------+ | | v | v_d | v_r | Angstrom^3 | +----------+------------+------------------+------------------+------------+ | Vectors | a, b, c | a_v, b_v, c_v | a_w, b_w, c_w | Angstrom | | +------------+------------------+------------------+------------+ | | x, y, z | x_v, y_v, z_v | x_w, y_w, z_w | Angstrom | +----------+------------+------------------+------------------+------------+ | Matrices | A | A_d | A_r | Angstrom^2 | | +------------+------------------+------------------+------------+ | | G | G_d | G_r | Angstrom^2 | +----------+------------+------------------+------------------+------------+ .. py:attribute:: IMPORTED_FROM_CIF .. py:attribute:: orientation 3x3 matrix describing orientation of crystal during experiment. .. py:method:: edit_cell(**parameters) Edit direct space unit cell using a dictionary with the following keys: - "a" - for unit cell parameter *a* given in Angstrom, - "b" - for unit cell parameter *b* given in Angstrom, - "c" - for unit cell parameter *c* given in Angstrom, - "al" - for unit cell parameter *alpha* given in degrees or radians, - "be" - for unit cell parameter *beta* given in degrees or radians, - "ga" - for unit cell parameter *gamma* given in degrees or radians. This method is equivalent to manually setting all six unit cell parameters in direct space, :attr:`a_d`, :attr:`b_d`, :attr:`c_d`, :attr:`al_d`, :attr:`be_d`, :attr:`ga_d`, and then running a private method :meth:`_refresh_cell` to update other values. Please mind that the while the "a", "b" and "c" are always given in Angstrom, the angles might be given either in degrees or in radians. For details see function :func:`hikari.utility.math_tools.angle2rad`. It is not required for all previously stated keys to be present at each method call. If a key has not been given, previously provided and stored value is being used. If no value has been given, the default length values of 1.0 for *a*, *b*, *c* and default angle values of *pi/2* for *al*, *be*, *ga* are used instead. :param parameters: Values of unit cell parameters to be changed :type parameters: float .. py:method:: fill_from_cif_block(block, fragile=False) Import all data specified in :attr:`~.BaseFrame.IMPORTED_FROM_CIF` such as unit cell parameters and orientation matrix from provided instance of :class:`hikari.dataframes.cif.CifBlock` called `block`. Unless `fragile` is `True`, use defaults instead of rising KeyError. :param block: CifBlock containing imported information. :type block: hikari.dataframes.CifBlock :param fragile: If True, raise Error when any imported info is missing :type fragile: bool .. py:method:: _refresh_cell() Recalculate all vectors and scalars other than :attr:`a_d`, :attr:`b_d`, :attr:`c_d`, :attr:`al_d`, :attr:`be_d`, :attr:`ga_d` based on the currently stored values of the aforementioned six. .. py:property:: a_d Length of unit cell vector **a** in direct space. :rtype: float :type: return .. py:property:: b_d Length of unit cell vector **b** in direct space. :rtype: float :type: return .. py:property:: c_d Length of unit cell vector **c** in direct space. :rtype: float :type: return .. py:property:: al_d Angle between vectors **b** and **c** in degrees. :rtype: float :type: return .. py:property:: be_d Angle between vectors **c** and **a** in degrees. :rtype: float :type: return .. py:property:: ga_d Angle between vectors **a** and **b** in degrees. :rtype: float :type: return .. py:property:: v_d Unit cell volume in direct space. :rtype: float :type: return .. py:property:: a_v Unit cell vector **a** in direct space. :rtype: numpy.array :type: return .. py:property:: b_v Unit cell vector **b** in direct space. :rtype: numpy.array :type: return .. py:property:: c_v Unit cell vector **c** in direct space. :rtype: numpy.array :type: return .. py:property:: A_d Basis matrix A with vertically stacked direct space vectors. :rtype: np.array :type: return .. py:property:: G_d Direct space metric matrix [ai . aj]ij. :rtype: np.array :type: return .. py:property:: a_r Length of unit cell vector **a\*** in reciprocal space. :rtype: float :type: return .. py:property:: b_r Length of unit cell vector **b\*** in reciprocal space. :rtype: float :type: return .. py:property:: c_r Length of unit cell vector **c\*** in reciprocal space. :rtype: float :type: return .. py:property:: al_r Angle between vectors **b\*** and **c\*** in degrees. :rtype: float :type: return .. py:property:: be_r Angle between vectors **c\*** and **a\*** in degrees. :rtype: float :type: return .. py:property:: ga_r Angle between vectors **a\*** and **b\*** in degrees. :rtype: float :type: return .. py:property:: v_r Unit cell volume in reciprocal space. :rtype: float :type: return .. py:property:: a_w Unit cell vector **a\*** in reciprocal space. :rtype: numpy.array :type: return .. py:property:: b_w Unit cell vector **b\*** in reciprocal space. :rtype: numpy.array :type: return .. py:property:: c_w Unit cell vector **c\*** in reciprocal space. :rtype: numpy.array :type: return .. py:property:: A_r Basis matrix A\* with vertically stacked reciprocal space vectors. :rtype: np.array :type: return .. py:property:: G_r Reciprocal space metric matrix [ai\* . aj\*]ij. :rtype: np.array :type: return .. py:attribute:: SELLING_S6_TRANSFORMATIONS .. py:attribute:: SELLING_E3_TRANSFORMATIONS .. py:class:: UBaseFrame Bases: :py:obj:`hikari.dataframes.BaseFrame` A sub-class of :class:`hikari.dataframes.BaseFrame` capable of the same operation as its parent, but using `uncertainty.ufloats` instead of floats. As a result, types specified in docstring might be wrong due to inheritance. .. py:attribute:: IMPORTED_FROM_CIF .. py:attribute:: orientation 3x3 matrix describing orientation of crystal during experiment. .. py:method:: _refresh_cell() Recalculate all vectors and scalars other than :attr:`a_d`, :attr:`b_d`, :attr:`c_d`, :attr:`al_d`, :attr:`be_d`, :attr:`ga_d` based on the currently stored values of the aforementioned six. .. py:class:: HklFrame Bases: :py:obj:`hikari.dataframes.BaseFrame` A master object which manages single-crystal diffraction files. It utilises other `Hkl*` classes to import, store, manipulate and output information about single-crystal diffraction patterns. HklFrame acts as a container which stores the diffraction data (Pandas dataframe, :attr:`table`) and elementary crystal cell data (:class:`hikari.dataframes.Base`). Demanding methods belonging to this class are vectorized, providing relatively satisfactory performance and high memory capacity. HklFrame methods are designed to work in-place, so the work strategy is to create a new instance of HklFrame for each reflection dataset, manipulate it using methods, eg. :func:`merge` or :func:`trim`, and :func:`copy` to other object or output using :func:`write` if needed. The HklFrame always initiates empty and does not accept any arguments. Some magic methods, such as :func:`__len__` and :func:`__add__` are defined and describe/operate on the :attr:`frame`. .. py:attribute:: HKL_LIMIT :value: 127 Highest absolute value of h, k or l index, which can be interpreted correctly by current version of the software. .. py:attribute:: __la :value: 0.71069 Wavelength of radiation used in experiment. .. py:attribute:: table Pandas dataframe containing diffraction data information. Each row represents one reflection observation, while each column has one piece of information about the reflections. For a list of available keys, see :class:`HklKeys`, whose instance is used to menage the keys of this table. .. py:method:: __add__(other) :param other: HklFrame to be added to data :type other: HklFrame :return: concatenated :attr:`table` dataframes with metadata from first :rtype: HklFrame .. py:method:: __len__() :return: Number of rows (individual reflections) in `self.data` :rtype: int .. py:method:: __str__() :return: Human-readable representation of `self.data` :rtype: str .. py:property:: la Wavelength of radiation used in the diffraction experiment. Can be set using popular abbreviations such as "MoKa" or "CuKb", where *a* and *b* stand for *alpha* and *beta*. Implemented cathode materials include: "Ag", "Co", "Cr", "Cu", "Fe", "Mn", "Mo", "Ni", "Pd", "Rh", "Ti", "Zn" and have been imported from International Tables of Crystallography, Volume C, Table 4.2.4.1, 3rd Edition. :return: wavelength of radiation used in experiment :rtype: float .. py:property:: r_lim Radius of limiting sphere in A^-1 calculated as 2/:attr:`la` :rtype: float :type: return .. py:method:: _in_dacs(opening_angle, vectors) .. py:method:: dac_trim(opening_angle = 35.0, vector=None) Remove reflections outside the opening_angle DAC-accessible volume. Sample/DAC orientation can be supplied either via specifying crystal orientation in :class:`hikari.dataframes.BaseFrame`, in :attr:`orientation` or providing a xyz\* *vector* perpendicular to the dac-accessible disc. For further details, see `*Tchoń & Makal, IUCrJ 8, 1006-1017 (2021)* `_. :param opening_angle: DAC single opening angle in degrees, default 35. :type opening_angle: float :param vector: Provides information about orientation of crystal relative to DAC. If None, :attr:`orientation` is used instead. :type vector: tuple[float] :return: HklFrame containing only reflections in dac-accessible region. :rtype: HklFrame .. py:method:: dacs_count(opening_angle = 35.0, vectors = np.array((1, 0, 0))) Count unique dac-accessible reflections for n crystals placed such that vector n is perpendicular to diamond. For details see :meth:`dac_trim`. :param opening_angle: DAC single opening angle in degrees, default 35. :type opening_angle: float :param vectors: Array with rotational axes of available DAC-discs. :type vectors: np.array :return: Array with numbers of unique reflns in DAC-accessible region. :rtype: np.array .. py:method:: copy() :return: An exact deep copy of this HklFrame. :rtype: HklFrame .. py:method:: extinct(space_group = SG['P1']) Removes from dataframe reflections which should be extinct based on space :class:`hikari.symmetry.group.Group`. For ref. see ITC-A12.3.5. :param space_group: Space group used to extinct the reflections. :type space_group: hikari.symmetry.group.Group .. py:method:: find_equivalents(point_group = PG['1']) Assign each reflection its symmetry equivalence identifier and store it in the `hikari.dataframes.HklFrame.data['equiv']` column. The ID is an integer unique for each set of equivalent reflections. In order to provide an information about equivalence, a *point_group* of reciprocal space must be provided (default PG['1']). Point groups and their notation can be found in :mod:`hikari.symmetry` sub-package. :param point_group: Point group used to determine symmetry equivalence :type point_group: hikari.symmetry.Group .. py:method:: from_dict(dictionary) Construct the `self.data` using information stored in dictionary. The dictionary keys must be valid strings, see :class:`HklKeys` for a list of valid keys. The dictionary values must be iterable of equal size, preferably `numpy.ndarray`. :param dictionary: Dictionary with "key - iterable of values" pairs. :type dictionary: Dict[str, numpy.ndarray] .. py:method:: fill(radius = 2.0) Fill dataframe with all reflections within *radius* from space origin. :param radius: Maximum distance from the reciprocal space origin to placed reflection (in reciprocal Angstrom). .. py:method:: stats(bins = 10, space_group = SG['P1']) Returns completeness, redundancy, number of all, unique & theoretically possible reflections within equal-volume `bins` in given `space group`. :param bins: Number of equal-volume bins to divide the data into. :type bins: int :param space_group: Group used to calculate equivalence and extinctions :type space_group: hikari.symmetry.Group :return: String containing table with stats as a function of resolution :rtype: str .. py:method:: merge(point_group=PG['1']) Average down each set of redundant reflections present in the table, to one reflection. The redundancy is determined using the :meth:`find_equivalents` method with appropriate point group. Thus, the merging can be used in different ways depending on point group: - For PG['1'], only reflections with exactly the same h, k, l indices will be merged. Resulting dataframe will not contain any duplicates. - For PG['-1'] reflections with the same h, k and l as well as their Friedel pairs will be merged together to one reflection. - For PG['mmm'] all equivalent reflections of "mmm" point group will be merged. Since "mmm" is centrosymmetric, Friedel pairs will be merged. - For PG['mm2'] symmetry-equivalent reflections within the "mmm" point group will be merged, but the Friedel pairs will be preserved. The procedure will have a different effect on different dataframe keys, depending on their "reduce_behaviour" specified in :class:`HklKeys`. Fixed parameters *h, k, l, x, y, z, r* and *equiv* will be preserved; Floating points such as intensity *I*, structure factor *F* and their uncertainties *si* and *sf* will be averaged using arithmetic mean; Multiplicity *m* will be summed; Other parameters which would lose their meaning such as batch number *b* will be discarded. The merging inevitably removes some information from the dataframe, but it can be necessary for some operations. For example, the drawing procedures work faster and provide clearer image if multiple points occupying the same position in space are reduced to one instance. :param point_group: Point Group used to determine symmetry equivalence :type point_group: hikari.symmetry.Group .. py:method:: place() Assign reflections their positions in reciprocal space ("x", "y", "z") and calculate their distance from origin ("r") in reciprocal Angstrom. Save four new keys and their values into the dataframe. .. py:method:: calculate_fcf_statistics() Calculate values of zeta (I - Ic) / si on other stats based on contents of fcf files. Save new key and its values into the dataframe. .. py:method:: read(hkl_path, hkl_format='shelx_4') Read the contents of .hkl file as specified by path and format, and store them in the pandas dataframe in `self.data`. For a list of all available .hkl formats, please refer to :attr:`hikari.dataframes.HklIo.format`. :param hkl_path: Absolute or relative path to the .hkl file. :type hkl_path: str :param hkl_format: Format of provided .hkl file. :type hkl_format: union[int, str, dict] .. py:method:: _recalculate_structure_factors_and_intensities() Calculate 'I' and 'si' or 'F' and 'sf', depending on which are missing. .. py:method:: _recalculate_structure_factors_from_intensities() Recalculate the structure factor F and its uncertainty sf. Structure factor is calculated as follows: *F = signum(I) \* sqrt(abs(I))*. Structure factor's uncertainty is calculated as follows: *sf = si / (2 \* sqrt(abs(I)))*. .. py:method:: _recalculate_intensities_from_structure_factors() Recalculate the intensity I and its uncertainty si. Intensity is calculated as follows: *I = signum(F) \* F \*\* 2*. Intensity's uncertainty is calculated as follows: *si = 2 \* sf \* abs(F)*. .. py:method:: transform(operations) Apply a symmetry operation or list of symmetry operations to transform the diffraction pattern. If one symmetry operation (3x3 or 4x4 numpy array) is provided, it effectively multiplies the hkl matrix by the operation matrix and accordingly alters the `self.data` dataframe. As a result, the length of `self.data` before and after transformation is the same. However, the function behaves slightly counter-intuitively if two or more operation matrices are provided. In such case the method applies the transformation procedure independently for each operation, and then *concatenates* resulting matrices. Resulting self.data is len(operations) times longer than the initial. The function can use 3x3 or larger (e.g. 4x4) matrices, as it selects only the upper-left 3x3 segment for the sake of calculations. Also, while reconstructing the symmetry of merged reflection file it is important to use all symmetry operations, not only generators. Single symmetry operations or their lists belonging to certain point groups can be imported from :py:mod:`hikari.symmetry` module. :param operations: Iterable of operation matrices to be applied :type operations: Union[Iterable[np.ndarray], np.ndarray] .. py:method:: thin_out(target_cplt=1.0) Randomly remove reflections from dataframe in order to decrease the completeness to *target_cplt* (relatively to initial completeness). :param target_cplt: Percentage of data not removed from dataframe :type target_cplt: float .. py:method:: to_res(path='hkl.res', colored='m') Export the reflection information from table to .res file, so that a software used to visualize .res files can be used to visualize a diffraction data in three dimensions. :param colored: Which key of dataframe should be visualized using color :type colored: str :param path: Absolute or relative path where the file should be saved :type path: str .. py:method:: trim(limit) Remove reflections further than *limit* from reciprocal space origin. :param limit: Radius of the trimming sphere in reciprocal Angstrom :type limit: float .. py:method:: write(hkl_path, hkl_format='shelx_4') Write the contents of dataframe to a .hkl file using specified *path* and *format*. For a list of all available .hkl formats, please refer to :attr:`hikari.dataframes.HklIo.format`. :param hkl_path: Absolute or relative path to the .hkl file. :type hkl_path: str :param hkl_format: Desired format of .hkl file. :type hkl_format: union[int, str, dict] .. py:class:: ResFrame Bases: :py:obj:`hikari.dataframes.BaseFrame` This class stores and manipulates basic information present in majority of crystallographic information files such as unit cell parameters stored in scalars and vectors. BaseFrame utilises the following notation for stored attributes: - The name begins from a unit cell property we are interested in: - "a", "b", "c" describe unit cell lengths/vectors *a*, *b*, *c*, - "al", "be", "ga" describe unit cell angles *alpha*, *beta*, *gamma*, - "v" describes unit cell volume, - "x", "y", "z" describe directions - normalised unit cell vectors. - "A", "G" describe stacked vector and metric matrix, respectively. - The unit cell parameter symbol is then followed by an underscore "_". - The name ends with a single letter denoting type of space and variable: - "d" (from Direct) denotes direct space scalars/matrices, - "r" (from Reciprocal) denotes reciprocal space scalars/matrices, - "v" (from Vector) denotes direct space vectors, - 'w" (similar to "v") denotes reciprocal space vectors. The values can be accessed by referencing a given attribute in the object, for example :class:`BaseFrame`. :attr:`a_d` stores information about the lattice constant *a* in direct space as a floating point, but :class:`BaseFrame`. :attr:`a_v` is a direct space vector. Available attributes have been once again presented in a table below: +----------+------------+------------------+------------------+------------+ | | Available | in direct | in reciprocal |Unit (^-1 in| | | constants | space | space |reciprocal) | +==========+============+==================+==================+============+ | Scalars | a, b, c | a_d, b_d, c_d | a_r, b_r, c_r | Angstrom | | +------------+------------------+------------------+------------+ | | al, be, ga | al_d, be_d, ga_d | al_r, be_r, ga_r | Radian | | +------------+------------------+------------------+------------+ | | v | v_d | v_r | Angstrom^3 | +----------+------------+------------------+------------------+------------+ | Vectors | a, b, c | a_v, b_v, c_v | a_w, b_w, c_w | Angstrom | | +------------+------------------+------------------+------------+ | | x, y, z | x_v, y_v, z_v | x_w, y_w, z_w | Angstrom | +----------+------------+------------------+------------------+------------+ | Matrices | A | A_d | A_r | Angstrom^2 | | +------------+------------------+------------------+------------+ | | G | G_d | G_r | Angstrom^2 | +----------+------------+------------------+------------------+------------+ .. py:attribute:: data .. py:method:: atomic_form_factor(atom, hkl) Calculate X-ray atomic form factors for a single atom and a hkl array :param atom: Atom/ion name/identifier interpreted by form factor table :type atom: str :param hkl: A 2D array listing all hkls to consider :type hkl: np.array :return: A 1D array listing atomic form factors for desired hkls :rtype: np.array .. py:method:: temperature_factor(hkl, u) Calculate temperature factor for single u matrix and a hkl array :param hkl: A 2D array listing all hkls to consider :type hkl: np.array :param u: A classical anisotropic displacement parameters matrix :type u: np.array :return: A 1D array listing temperature factors for desired hkls :rtype: np.array .. py:method:: form_factor(hkl, space_group) Calculate form factors based on current structure, hkls, and space group :param hkl: A 2D array listing all hkls to consider :type hkl: np.array :param space_group: Space group describing the internal crystal symmetry :type space_group: hikari.symmetry.Group :return: A 1D array listing total form factors for desired hkls :rtype: np.array .. py:method:: read(path) Read data from specified ins/res file and return an dict :param path: Relative or absolute path to the res file to be read :type path: str :return: None :rtype: None .. py:class:: LstFrame .. py:method:: read_r1(path) :staticmethod: Read and return the final value of R1 from lst file