I performed a test in which I wrote four files to a tape using four parallel threads. The four threads use a lock around the write() system call to ensure that the drive is accessed serially. However, operations such as open(), fsync(), and close() for each file are not protected by the lock.
After all four jobs completed, I called ltfs.sync() once.
During the writing of the second file, a "Cannot write block" error occurred. The tape entered a permanent write error state, and LTFS immediately performed an extent cleanup. The relevant LTFS log is as follows:
Sep 15 01:54:53
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 64 LTFS11337I Update index-dirty
flag (1) - NO_BARCODE (0x0x55c5635ea710).
Sep 15 01:55:29
vdm3000-udm-utdmagent-0 user.err ltfs[98]: 65 LTFS12045E Cannot write block:
backend call failed (-20301). Dropping to read-only mode.
Sep 15 01:55:29
vdm3000-udm-utdmagent-0 user.err ltfs[98]: 65 LTFS11072E Cannot write blocks:
failed to write to the medium (-20301).
Sep 15 01:55:29
vdm3000-udm-utdmagent-0 user.err ltfs[98]: 65 LTFS11077E Cannot write: failed
to write blocks to the medium (-20301).
Sep 15 01:55:29
vdm3000-udm-utdmagent-0 user.warn ltfs[98]: 65 LTFS13014W Data partition
writer: failed to write data to the tape (-20301).
Sep 15 01:55:29
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS13024I Clean up extents and
append index at index partition (-20301).
Sep 15 01:55:31
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS17292I Current position is
(1, 10246), Error position is (1, 10117).
Sep 15 01:55:31
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS13025I Truncate extents
larger than position (1, 10117), block size = 524288.
Sep 15 01:55:31
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS11334I Remove extent :
utdm.0.0.1613966600.2147720233 (9407, 439877632).
Sep 15 01:55:31
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS11343I Try to write an index
on the IP on NO_BARCODE because of a permanent write error on the DP..
Sep 15 01:55:49
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS17235I Writing index of
NO_BARCODE to a (Reason: Write perm, 5 files) 11C9D0D0D3.
Sep 15 01:55:58
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS17236I Wrote index of
NO_BARCODE (Gen = 2, Part = a, Pos = 8, 11C9D0D0D3).
Sep 15 01:55:58
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS11337I Update index-dirty
flag (0) - NO_BARCODE (0x0x55c5635ea710).
Sep 15 01:55:58
vdm3000-udm-utdmagent-0 user.err ltfs[98]: 8c LTFS14022E Cannot flush file data
to the medium. Data might be lost (ltfs_fuse_flush).
On the application side, I observed that the write() calls for the second, third, and fourth files returned errno = 2. Then ltfs.sync() returned -1.
In this situation, LTFS performed an extent cleanup after the permanent write error. My questions are:
-
Can the contents of the first file, for which some write() calls had already returned success, be partially or completely affected by the cleanup?
In other words, is it possible that some data from the first file that had previously been reported as successfully written by write() was truncated or removed because of the subsequent permanent write error?
-
How can an upper-layer application reliably detect that the tape has entered the permanent write error ("write-perm") state?
Is there an LTFS API, ioctl, status interface, return code, or other mechanism that an application can use to detect this state without parsing the LTFS log?
-
How can the upper-layer application determine that LTFS has performed an extent cleanup or rollback?
Is there any API, event, status, or other mechanism that exposes this information to the application?
-
What should the upper-layer application do after ltfs.sync() returns -1?
What checks are recommended to determine whether previously successful writes are still valid and whether any data may have been lost or truncated?
-
More generally, what guarantees does LTFS provide for a successful write() in the presence of a subsequent permanent write error and extent cleanup?
-
It is quite possible that the "Cannot write block" error was caused by a drive-related problem. In that case, how can we reliably determine whether the root cause was the drive itself, rather than the tape medium or the I/O path (e.g., HBA/FC/backend)?
I would like to understand the expected semantics in this situation and the recommended way for an upper-layer application to detect and safely handle a permanent write error and the resulting extent cleanup.
I performed a test in which I wrote four files to a tape using four parallel threads. The four threads use a lock around the
write()system call to ensure that the drive is accessed serially. However, operations such asopen(),fsync(), andclose()for each file are not protected by the lock.After all four jobs completed, I called
ltfs.sync()once.During the writing of the second file, a "Cannot write block" error occurred. The tape entered a permanent write error state, and LTFS immediately performed an extent cleanup. The relevant LTFS log is as follows:
On the application side, I observed that the
write()calls for the second, third, and fourth files returnederrno = 2. Thenltfs.sync()returned-1.In this situation, LTFS performed an extent cleanup after the permanent write error. My questions are:
Can the contents of the first file, for which some
write()calls had already returned success, be partially or completely affected by the cleanup?In other words, is it possible that some data from the first file that had previously been reported as successfully written by
write()was truncated or removed because of the subsequent permanent write error?How can an upper-layer application reliably detect that the tape has entered the permanent write error ("write-perm") state?
Is there an LTFS API, ioctl, status interface, return code, or other mechanism that an application can use to detect this state without parsing the LTFS log?
How can the upper-layer application determine that LTFS has performed an extent cleanup or rollback?
Is there any API, event, status, or other mechanism that exposes this information to the application?
What should the upper-layer application do after
ltfs.sync()returns-1?What checks are recommended to determine whether previously successful writes are still valid and whether any data may have been lost or truncated?
More generally, what guarantees does LTFS provide for a successful
write()in the presence of a subsequent permanent write error and extent cleanup?It is quite possible that the "Cannot write block" error was caused by a drive-related problem. In that case, how can we reliably determine whether the root cause was the drive itself, rather than the tape medium or the I/O path (e.g., HBA/FC/backend)?
I would like to understand the expected semantics in this situation and the recommended way for an upper-layer application to detect and safely handle a permanent write error and the resulting extent cleanup.