IMCtermite provides access to proprietary IMC measurement file formats introduced and developed by imc Test & Measurement GmbH. The library currently supports both the legacy IMC2 Data Format and the newer IMC3 Data Format, which are commonly encountered with the file extensions .raw and .dat. These formats are employed i.a. by the measurement hardware imc CRONOSflex to dump and store data and the software packages imc Studio & imc FAMOS for measurement data control and analysis. Thanks to the integrated Python module, the extracted measurement data can be stored in any open-source file format accessible by Python like i.a. csv, json or parquet.
On the Record Evolution Platform, the library can be used both as a command line tool for interactive usage and as a Python module to integrate IMC measurement files into any ETL workflow.
- IMC2 marker-based files. Reference: docs/imc2-format.md
- IMC3 files, including bundled and single-channel variants. Reference: docs/imc3-format.md
IMCtermite auto-detects the file format while loading a file. The same CLI and Python interfaces are used for both supported formats; no explicit format flag is required.
| format | typical extensions | support status | notes |
|---|---|---|---|
| IMC2 | .raw, .dat | supported | Legacy marker-based stream format |
| IMC3 | .raw, .dat | supported | Structured successor format; compressed IMC3 is currently rejected |
Detailed format notes live in dedicated pages:
The IMCtermite library may be employed both as a CLI tool and a python module.
To build the CLI tool locally, use the default target make resulting
in the binary imctermite. To ensure system-wide availability, the installation
of the tool (in the default location /usr/local/bin) is done via
make install
which may require root permissions.
To integrate the library into a customized ETL toolchain, several python targets are available. For a local in-place build that enables you to run the examples, use:
make python-build
The package is also available in the Python Package Index at imctermite. To install the latest version simply do
python3 -m pip install imctermitewhich provides binary wheels for multiple architectures on Windows and Linux and most Python 3.x distributions. Note: Starting from version 3.0.0, imctermite requires numpy as a dependency, which will be automatically installed if not already present.
However, if your platform/architecture is not supported you can still compile the source distribution yourself, which requires python3_setuptools, numpy, and an up-to-date compiler supporting C++11 standard (e.g. gcc version >= 10.2.0).
The usage of the imctermite binary looks like this:
imctermite <imc-file> [options]
You have to provide a single supported IMC file and any option to specify what
to do with the data. All available options can be listed with imctermite --help:
Options:
-c, --listchannels list channels
-b, --listblocks list IMC key-blocks
-d, --output output directory to print channels
-s, --delimiter csv delimiter/separator char for output
-h, --help show this help message
-v, --version display version
For instance, to show a list of all channels included in sample-data.raw, you
do imctermite sample-data.raw --listchannels. No output files are
written by default. Output files are written only when an existing (!) directory
is provided as argument to the --output option. By default, every output file
is written using a , delimiter. You may provide any custom separator with the
option --delimiter. For example, in order to use |, the binary is called with
options imctermite sample-data.raw -b -c -s '|'.
Given the IMCtermite module is available, we can import it and declare an instance
of it by passing an IMC file to the constructor:
from imctermite import ImcTermite
imcraw = ImcTermite("sample/sampleA.raw")An example of how to create an instance and obtain the list of channels is:
from imctermite import ImcTermite
# declare and initialize instance of "imctermite" by passing an IMC file
try :
imcraw = ImcTermite("samples/sampleA.raw")
except RuntimeError as e :
print("failed to load/parse IMC file: " + str(e))
# obtain list of channels as list of dictionaries (without data)
channels = imcraw.get_channels(False)
print(channels)A more complete example, including the methods for
obtaining the channels, i.a. their data and/or directly printing them to files,
can be found in the python/examples folder.
For large files, you can iterate over channel data in chunks as NumPy arrays. This avoids creating large Python lists and allows for streaming processing (e.g. writing to Parquet). See python/examples/usage_numpy_chunks.py for a complete example.
Prepare the local test environment with make prepare-test, then run end-to-end tests with make test.
The Python environment is expected to be prepared outside Make and already contain the required build and test dependencies.
To emit the Python wrapper coverage report through the same target, ensure pytest-cov is installed and use make test COVERAGE=1.
See tests/README.md for details.
- https://www.imc-tm.de/produkte/messtechnik-software/imc-famos/import-export
- https://www.imc-tm.de/produkte/messtechnik-hardware/imc-cronosflex/ueberblick/
- https://www.imc-tm.de/download-center/produkt-downloads/imc-famos/handbuecher
- https://www.imc-tm.de/fileadmin/Public/Downloads/Manuals/imc_FAMOS/imcGemeinsameKomponenten.pdf
- https://github.com/Apollo3zehn/ImcFamosFile
- https://apollo3zehn.github.io/ImcFamosFile/api/ImcFamosFile.FamosFileKeyType.html
- https://pypi.org/help/#apitoken
- https://sgoel.dev/posts/uploading-binary-wheels-to-pypi-from-github-actions/
- https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstepsrun
- https://github.com/pypa/cibuildwheel/blob/main/examples/github-deploy.yml
- https://cibuildwheel.readthedocs.io/en/stable/deliver-to-pypi/
- https://github.com/actions/download-artifact#download-all-artifacts
- https://github.com/actions/download-artifact?tab=readme-ov-file#download-multiple-filtered-artifacts-to-the-same-directory