Describe the bug
Currently, when attempting to do in-place processing for VRL files with RDWs, you get the following error. This is because this mode does not have a built-in record extractor.
Code snippet that caused the issue
df.write
.format("cobol")
.option("record_format","V")
.option("is_rdw_big_endian", "true")
.option("copybook_contents", copybook)
.save(outputPath)
The error is:
java.lang.IllegalArgumentException: Cannot create a record extractor for the given reader parameters. Please check the copybook and the reader parameters.
at za.co.absa.cobrix.cobol.processor.impl.CobolProcessorBase$.getRecordExtractor(CobolProcessorBase.scala:46)
at za.co.absa.cobrix.spark.cobol.SparkCobolProcessor$.$anonfun$getRecordRdd$2(SparkCobolProcessor.scala:240)
Expected behavior
This format should be supported for in-place processing.
Context
- Cobrix version: 2.11.0
- Spark version: 3.5.4
- Scala version: 2.12
Copybook (if possible)
Describe the bug
Currently, when attempting to do in-place processing for VRL files with RDWs, you get the following error. This is because this mode does not have a built-in record extractor.
Code snippet that caused the issue
df.write .format("cobol") .option("record_format","V") .option("is_rdw_big_endian", "true") .option("copybook_contents", copybook) .save(outputPath)The error is:
Expected behavior
This format should be supported for in-place processing.
Context
Copybook (if possible)