Skip to content

Commit f2d31fd

Browse files
yutaro-sakamotoCopilot
yutaro-sakamoto
and
Copilot
authored
Add cobj-idx migrate and cobj-idx unlock (#693)
* add: processUuid and processId * add: create a new table `file_lock` * refactor: suppress PMD warnings * fix: sql statement creating a new table `file_lock` * wip: wip * fix: a PMD warning * fix: a SpotBug warning * refactor: remove printStackTrace() * fix: check formats of indexed files to open * test: remove unnecessary checks * fix: change constants to avoid overlaps * fix: add a file existing check * test: fix the test of opening an invalid indexed file * test: fix load.at not to call default error handlers * chore: end a transaction * refactor: CobolIndexedFile.java * test: prevent infinite loops * refactor: CobolIndexedFile.java * wip: wip * fix: improve open_ in CobolIndexedFile.java * change: set transaction settings * set the transaction mode to EXCLUSIVE * set busy_timeout to 5000 * feat: implement file locks of indexed files * fix: change the condition of creating file_lock * change: set the transaction isolation to SERIALIZABLE * test: add tests of file locking of indexed files * change: the schema of the record key table * fix: an INSERT statement * fix: an insert statement for primary record tables * feat: add a new option `-default_select_lock_mode` * channge: insert information when calling read_() * fix: the method read_ * Implement file locking of indexed files (#678) * add: processUuid and processId * add: create a new table `file_lock` * refactor: suppress PMD warnings * fix: sql statement creating a new table `file_lock` * wip: wip * fix: a PMD warning * fix: a SpotBug warning * refactor: remove printStackTrace() * fix: check formats of indexed files to open * test: remove unnecessary checks * fix: change constants to avoid overlaps * fix: add a file existing check * test: fix the test of opening an invalid indexed file * test: fix load.at not to call default error handlers * chore: end a transaction * refactor: CobolIndexedFile.java * test: prevent infinite loops * refactor: CobolIndexedFile.java * wip: wip * fix: improve open_ in CobolIndexedFile.java * change: set transaction settings * set the transaction mode to EXCLUSIVE * set busy_timeout to 5000 * feat: implement file locks of indexed files * fix: change the condition of creating file_lock * change: set the transaction isolation to SERIALIZABLE * test: add tests of file locking of indexed files * Update libcobj/app/src/main/java/jp/osscons/opensourcecobol/libcobj/file/CobolIndexedFile.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update libcobj/app/src/main/java/jp/osscons/opensourcecobol/libcobj/file/CobolIndexedFile.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * refactor: remove unnecessary comments * refactor: fix typo * refactor: fix typo --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * refactor: CobolIndexedFile.java * add: lock a record when READ NEXT is executed * add: unlock records when closing indexed files * fix: the process of unlocking a file lock * fix: controling transactions when rewriting records * fix: commit timing when writing records * fix: commits and rollbacks in WRITE, REWRITE and DELETE * change: -default_lock_mode to -lock-mode-automatic * refactor: code for commiting data * add: unlock the previous record after writing * add: unlock previous records after rewriting * add: impelment record lockings of DELETE * fix: constant numbers change the value of COB_LOCK_AUTOMATIC and COB_LOCK_MULTIPLE in CobolFile.java * chore: remove duplicated code * fix: close ResultSet not to lock database implicitly * fix: close ResultSet in CobolIndexedFile.java * add: NewIndexedCursor.java * wip: read forward * wip: read prev * fix: fetchRecord * fix: READ NEXT/PREVIOUS * fix: READ * fix: fetchRecord * fix: fetchRecord * fix: START * fix: START ~ KEY IS GREATER THAN * fix: START ~ KEY IS <= * fix: START and READ NEXT RECORD * refactor: resolve PMD warnings * chore: migrate NewIndexedCursor to IndexedCursor * fix: allow READ WITH LOCK/WITH NO LOCK if LOCK MODE IS AUTOMATIC * fix: call commit() in the end of READ and READ NEXT/PREV * fix: initialize previousLockedReclordKey in close_() * test: add some tests for record locking * fix: unlock Previous records at the end of READ * test: accelerate file locking tests * test: wip * test: add tests for `READ WITH LOCK` * test: check the final state of an indexed file * fix: do not read data when the file status is not 00 * test: move common java code for record lock tests * test: add tests for different records * test: add tests for input mode * test: add tests for the same process * fix: commit at the end of start_ * test: guarantee that OPEN,START,WRITE,REWRITE cannot lock indexed file records * test: fix tests for OPEN,START,WRITE and REWRITE * fix: processes of unlocking records * tests: add tests for unlocking records * tests: add tests of OPEN INPUT * test: add tests for LOCK MODE * test: add tests of -lock-mode-automatic * tests: move tests for indexed files lock to tests/indexed-lock * ci: run tests `indexed-lock` * chore: fix typo * feat: add `cobj-idx migrate` * feat: add `cobj-idx unlock` * test: add a test for `cobj-idx migrate` * test: add a test for `cobj-idx unlock` * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * chore: simplify some `if` conditions --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent dbcfae7 commit f2d31fd

7 files changed

Lines changed: 351 additions & 1 deletion

File tree

‎libcobj/app/src/main/java/jp/osscons/opensourcecobol/libcobj/user_util/indexed_file/IndexedFileUtilMain.java‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -137,6 +137,40 @@ public static void main(String[] args) {
137137
System.err.println("error: " + e.getMessage());
138138
System.exit(1);
139139
}
140+
} else if ("migrate".equals(subCommand)) {
141+
if (unrecognizedArgs.length != 2) {
142+
if (unrecognizedArgs.length < 2) {
143+
System.err.println("error: no indexed file is specified.");
144+
} else {
145+
System.err.println("error: too many indexed files are specified.");
146+
}
147+
System.exit(1);
148+
}
149+
String indexedFilePath = unrecognizedArgs[1];
150+
try {
151+
migrateIndexedFile(indexedFilePath);
152+
} catch (Exception e) {
153+
System.err.println("error: " + e.getMessage());
154+
System.exit(1);
155+
}
156+
System.exit(0);
157+
} else if ("unlock".equals(subCommand)) {
158+
if (unrecognizedArgs.length != 2) {
159+
if (unrecognizedArgs.length < 2) {
160+
System.err.println("error: no indexed file is specified.");
161+
} else {
162+
System.err.println("error: too many indexed files are specified.");
163+
}
164+
System.exit(1);
165+
}
166+
String indexedFilePath = unrecognizedArgs[1];
167+
try {
168+
unlockIndexedFile(indexedFilePath);
169+
} catch (Exception e) {
170+
System.err.println("error: " + e.getMessage());
171+
System.exit(1);
172+
}
173+
System.exit(0);
140174
} else if ("load".equals(subCommand)) {
141175
if (unrecognizedArgs.length < 2 || unrecognizedArgs.length > 3) {
142176
if (unrecognizedArgs.length < 2) {
@@ -226,6 +260,14 @@ private static void printHelpMessage() {
226260
" Write the records stored in the indexed file into the output file.");
227261
System.out.println(" The default format of the output data is SEQUENTIAL of COBOL.");
228262
System.out.println();
263+
System.out.println("cobj-idx migrate <indexed file>");
264+
System.out.println(
265+
" Migrate the indexed file whose version is older than 1.1.12 to the latest"
266+
+ " version.");
267+
System.out.println();
268+
System.out.println("cobj-idx unlock <indexed file>");
269+
System.out.println(" Unlock all locks of the indexed file.");
270+
System.out.println();
229271
System.out.println("Options:");
230272
System.out.println();
231273
System.out.println("-f <format>, --format=<format>");
@@ -413,6 +455,60 @@ private static int processInfoCommand(String indexedFilePath) {
413455
}
414456
}
415457

458+
private static void migrateIndexedFile(String indexedFilePath) throws Exception {
459+
try (Connection conn = DriverManager.getConnection("jdbc:sqlite:" + indexedFilePath);
460+
Statement st = conn.createStatement()) {
461+
String createTableSql =
462+
"CREATE TABLE IF NOT EXISTS file_lock (locked_by text primary key,process_id"
463+
+ " text,locked_at timestamp,open_mode text CONSTRAINT check_open_mode"
464+
+ " CHECK (open_mode IN ('INPUT', 'OUTPUT', 'I-O', 'EXTEND')))";
465+
st.executeUpdate(createTableSql);
466+
467+
boolean lockedByColumnExists = false;
468+
boolean processIdColumnExists = false;
469+
boolean lockedAtColumnExists = false;
470+
try (ResultSet rs = st.executeQuery("PRAGMA table_info('table0')")) {
471+
while (rs.next()) {
472+
String columnName = rs.getString("name");
473+
if ("locked_by".equals(columnName)) {
474+
lockedByColumnExists = true;
475+
} else if ("process_id".equals(columnName)) {
476+
processIdColumnExists = true;
477+
} else if ("locked_at".equals(columnName)) {
478+
lockedAtColumnExists = true;
479+
}
480+
if (lockedByColumnExists && processIdColumnExists && lockedAtColumnExists) {
481+
break;
482+
}
483+
}
484+
}
485+
if (!lockedByColumnExists) {
486+
st.executeUpdate("ALTER TABLE table0 ADD COLUMN locked_by text");
487+
}
488+
if (!processIdColumnExists) {
489+
st.executeUpdate("ALTER TABLE table0 ADD COLUMN process_id text");
490+
}
491+
if (!lockedAtColumnExists) {
492+
st.executeUpdate("ALTER TABLE table0 ADD COLUMN locked_at timestamp");
493+
}
494+
} catch (SQLException e) {
495+
throw new Exception(
496+
String.format("failed to connect to the indexed file: %s", indexedFilePath), e);
497+
}
498+
}
499+
500+
private static void unlockIndexedFile(String indexedFilePath) throws Exception {
501+
try (Connection conn = DriverManager.getConnection("jdbc:sqlite:" + indexedFilePath);
502+
Statement st = conn.createStatement()) {
503+
st.executeUpdate("DELETE FROM file_lock");
504+
st.executeUpdate(
505+
"UPDATE table0 SET locked_by = NULL, process_id = NULL, locked_at = NULL");
506+
} catch (SQLException e) {
507+
throw new Exception(
508+
String.format("failed to connect to the indexed file: %s", indexedFilePath), e);
509+
}
510+
}
511+
416512
/**
417513
* Process load sub command, which loads data inputted from stdin to the indexed file.
418514
*

‎tests/Makefile.am‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -204,6 +204,8 @@ cobj_idx_DEPENDENCIES = \
204204
cobj-idx.src/info.at \
205205
cobj-idx.src/load.at \
206206
cobj-idx.src/unload.at \
207+
cobj-idx.src/migrate.at \
208+
cobj-idx.src/unlock.at \
207209
cobj-idx.src/misc.at
208210

209211
indexed_lock_DEPENDENCIES = \

‎tests/Makefile.in‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -743,6 +743,8 @@ cobj_idx_DEPENDENCIES = \
743743
cobj-idx.src/info.at \
744744
cobj-idx.src/load.at \
745745
cobj-idx.src/unload.at \
746+
cobj-idx.src/migrate.at \
747+
cobj-idx.src/unlock.at \
746748
cobj-idx.src/misc.at
747749

748750
indexed_lock_DEPENDENCIES = \

‎tests/cobj-idx.at‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,5 @@ m4_include([info.at])
55
m4_include([load.at])
66
m4_include([unload.at])
77
m4_include([cobj-idx.src/misc.at])
8-
8+
m4_include([migrate.at])
9+
m4_include([unlock.at])

‎tests/cobj-idx.src/migrate.at‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
AT_SETUP([migrate])
2+
3+
AT_DATA([check_file.cbl], [
4+
IDENTIFICATION DIVISION.
5+
PROGRAM-ID. check_file.
6+
7+
ENVIRONMENT DIVISION.
8+
INPUT-OUTPUT SECTION.
9+
FILE-CONTROL.
10+
SELECT F ASSIGN TO "indexed_file.dat"
11+
ORGANIZATION IS INDEXED
12+
ACCESS MODE IS DYNAMIC
13+
RECORD KEY IS REC-KEY
14+
ALTERNATE RECORD KEY IS REC-KEY2 WITH DUPLICATES
15+
FILE STATUS IS FILE-STATUS.
16+
17+
DATA DIVISION.
18+
FILE SECTION.
19+
FD f.
20+
01 REC.
21+
05 REC-KEY PIC X(5).
22+
05 REC-KEY2 PIC X(5).
23+
05 REC-DATA PIC X(5).
24+
25+
WORKING-STORAGE SECTION.
26+
01 FILE-STATUS PIC XX.
27+
01 DUMMY PIC X(10) VALUE "DUMMY DATA".
28+
PROCEDURE DIVISION.
29+
MAIN-PROCEDURE.
30+
31+
OPEN I-O f.
32+
33+
MOVE "AAAA1" TO REC-KEY.
34+
READ F WITH LOCK.
35+
DISPLAY REC.
36+
37+
MOVE "AAAA2" TO REC-KEY.
38+
REWRITE REC.
39+
40+
MOVE "AAAA2" TO REC-KEY.
41+
READ F WITH LOCK.
42+
DISPLAY REC.
43+
44+
MOVE "AAAA2" TO REC-KEY.
45+
DELETE F.
46+
DISPLAY FILE-STATUS.
47+
48+
MOVE "AAAA1" TO REC-KEY.
49+
START F KEY IS = REC-KEY.
50+
PERFORM FOREVER
51+
READ F NEXT RECORD
52+
AT END
53+
EXIT PERFORM
54+
NOT AT END
55+
DISPLAY REC
56+
END-READ
57+
END-PERFORM.
58+
CLOSE f.
59+
])
60+
61+
# copy the indexed file created with the older compiler to the current directory
62+
AT_CHECK([cp ../../cobj-idx.src/test-data/indexed_file.dat .])
63+
64+
# convert the indexed file to the current format
65+
AT_CHECK([${COBJ_IDX} migrate indexed_file.dat])
66+
67+
# check the converted file
68+
AT_CHECK([${COMPILE} check_file.cbl])
69+
AT_CHECK([java check_file], [0],
70+
[AAAA1BBBB1CCCC1
71+
AAAA2BBBB1CCCC1
72+
00
73+
AAAA1BBBB1CCCC1
74+
AAAA3BBBB3CCCC3
75+
])
76+
AT_CLEANUP

‎tests/cobj-idx.src/test-data/indexed_file.dat‎

40 KB
Binary file not shown.

‎tests/cobj-idx.src/unlock.at‎

Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
AT_SETUP([unlock])
2+
3+
# Create a sample indexed file named 'indexed_file.dat' and exit the program without closing the file
4+
AT_DATA([create_and_exit.cbl], [
5+
IDENTIFICATION DIVISION.
6+
PROGRAM-ID. create_and_exit.
7+
8+
ENVIRONMENT DIVISION.
9+
INPUT-OUTPUT SECTION.
10+
FILE-CONTROL.
11+
SELECT F ASSIGN TO "indexed_file.dat"
12+
ORGANIZATION IS INDEXED
13+
ACCESS MODE IS DYNAMIC
14+
RECORD KEY IS REC-KEY
15+
ALTERNATE RECORD KEY IS REC-KEY2 WITH DUPLICATES
16+
FILE STATUS IS FILE-STATUS.
17+
18+
DATA DIVISION.
19+
FILE SECTION.
20+
FD f.
21+
01 REC.
22+
05 REC-KEY PIC X(5).
23+
05 REC-KEY2 PIC X(5).
24+
05 REC-DATA PIC X(5).
25+
26+
WORKING-STORAGE SECTION.
27+
01 FILE-STATUS PIC XX.
28+
01 DUMMY PIC X(10) VALUE "DUMMY DATA".
29+
PROCEDURE DIVISION.
30+
MAIN-PROCEDURE.
31+
OPEN OUTPUT f.
32+
MOVE "AAAA1" TO REC-KEY.
33+
MOVE "BBBB1" TO REC-KEY2.
34+
MOVE "CCCC1" TO REC-DATA.
35+
WRITE REC.
36+
CLOSE f.
37+
38+
OPEN I-O f.
39+
MOVE "AAAA1" TO REC-KEY.
40+
READ f WITH LOCK.
41+
CALL "force_exit".
42+
])
43+
44+
AT_DATA([force_exit.java], [
45+
import java.io.UnsupportedEncodingException;
46+
import jp.osscons.opensourcecobol.libcobj.*;
47+
import jp.osscons.opensourcecobol.libcobj.common.*;
48+
import jp.osscons.opensourcecobol.libcobj.data.*;
49+
import jp.osscons.opensourcecobol.libcobj.exceptions.*;
50+
import jp.osscons.opensourcecobol.libcobj.termio.*;
51+
import jp.osscons.opensourcecobol.libcobj.call.*;
52+
import jp.osscons.opensourcecobol.libcobj.file.*;
53+
import jp.osscons.opensourcecobol.libcobj.ui.*;
54+
import java.util.Optional;
55+
56+
public class force_exit implements CobolRunnable {
57+
@Override
58+
public int run(CobolDataStorage... argStorages) {
59+
System.exit(0);
60+
return 0;
61+
}
62+
63+
@Override
64+
public void cancel() {
65+
return;
66+
}
67+
68+
@Override
69+
public boolean isActive() {
70+
return false;
71+
}
72+
}
73+
])
74+
75+
AT_DATA([check_file_lock.cbl],[
76+
IDENTIFICATION DIVISION.
77+
PROGRAM-ID. check_file_lock.
78+
79+
ENVIRONMENT DIVISION.
80+
INPUT-OUTPUT SECTION.
81+
FILE-CONTROL.
82+
SELECT F ASSIGN TO "indexed_file.dat"
83+
ORGANIZATION IS INDEXED
84+
ACCESS MODE IS DYNAMIC
85+
RECORD KEY IS REC-KEY
86+
ALTERNATE RECORD KEY IS REC-KEY2 WITH DUPLICATES
87+
FILE STATUS IS FILE-STATUS.
88+
89+
DATA DIVISION.
90+
FILE SECTION.
91+
FD f.
92+
01 REC.
93+
05 REC-KEY PIC X(5).
94+
05 REC-KEY2 PIC X(5).
95+
05 REC-DATA PIC X(5).
96+
97+
WORKING-STORAGE SECTION.
98+
01 FILE-STATUS PIC XX.
99+
01 DUMMY PIC X(10) VALUE "DUMMY DATA".
100+
PROCEDURE DIVISION.
101+
MAIN-PROCEDURE.
102+
OPEN OUTPUT f.
103+
DISPLAY FILE-STATUS.
104+
CLOSE f.
105+
])
106+
107+
AT_DATA([check_record_lock.cbl], [
108+
IDENTIFICATION DIVISION.
109+
PROGRAM-ID. check_record_lock.
110+
111+
ENVIRONMENT DIVISION.
112+
INPUT-OUTPUT SECTION.
113+
FILE-CONTROL.
114+
SELECT F ASSIGN TO "indexed_file.dat"
115+
ORGANIZATION IS INDEXED
116+
ACCESS MODE IS DYNAMIC
117+
RECORD KEY IS REC-KEY
118+
ALTERNATE RECORD KEY IS REC-KEY2 WITH DUPLICATES
119+
FILE STATUS IS FILE-STATUS.
120+
121+
DATA DIVISION.
122+
FILE SECTION.
123+
FD f.
124+
01 REC.
125+
05 REC-KEY PIC X(5).
126+
05 REC-KEY2 PIC X(5).
127+
05 REC-DATA PIC X(5).
128+
129+
WORKING-STORAGE SECTION.
130+
01 FILE-STATUS PIC XX.
131+
PROCEDURE DIVISION.
132+
MAIN-PROCEDURE.
133+
134+
OPEN I-O f.
135+
136+
MOVE "AAAA1" TO REC-KEY.
137+
READ F WITH LOCK.
138+
DISPLAY FILE-STATUS.
139+
140+
CLOSE f.
141+
])
142+
143+
AT_CHECK([${COMPILE} *.cbl])
144+
AT_CHECK([javac force_exit.java])
145+
146+
# Create the indexed file and exit the program without closing the file
147+
AT_CHECK([java create_and_exit])
148+
# Check the file is locked
149+
AT_CHECK([java check_file_lock], [0],
150+
[61
151+
])
152+
153+
AT_CHECK([rm -f indexed_file.dat])
154+
# Create the indexed file and exit the program without closing the file
155+
AT_CHECK([java create_and_exit])
156+
# Unlock the file
157+
AT_CHECK([${COBJ_IDX} unlock indexed_file.dat])
158+
# Check the file lock is released
159+
AT_CHECK([java check_file_lock], [0],
160+
[00
161+
])
162+
163+
AT_CHECK([rm -f indexed_file.dat])
164+
# Create the indexed file and exit the program without closing the file
165+
AT_CHECK([java create_and_exit])
166+
# Unlock the file
167+
AT_CHECK([${COBJ_IDX} unlock indexed_file.dat])
168+
# Check the record lock is released
169+
AT_CHECK([java check_record_lock], [0],
170+
[00
171+
])
172+
173+
AT_CLEANUP

0 commit comments

Comments
 (0)