Question

I am supposed to read from multiple files and then write into a single file after applying some processing and transforming logic. In order to do so, I have created a single batch job with 3 steps, where the 1st and 2nd steps are responsible for reading the data and writing in an intermediary temporary file, and finally in the 3rd or the last step I am trying to read that temporary file and apply the final logic to create the end file.

The error I am getting is:

Always the intermediate temporary file is getting created successfully as an end result of step 1 and step 2, but step 3 fails to create the final file.
Step 3 is able to find and read from that file and also was successful in preparing the data, but the file is never getting created.
Spring Batch is throwing a MojoException, and it is hard to find the actual cause.
For testing, if I keep the temporary file pre-created and directly run the last step, then the final file gets created successfully. But when all the steps are running and this temporary file is created on the fly, only then this error occurs.
I have tried with start-next steps and also with flow, i.e., start-on(COMPLTED)-to-end approach, but every time, the same results.
Finally, I have made transactional as false and forceSync as true in the writer but with no luck.

下面是作者的幻灯片和工作:

new FlatFileOutputWriter(
    new FlatFileItemWriterBuilder<MyPOJO>()
        .name("lastFileWriter")
        .resource(new FileSystemResource("<THE PATH TO THE TEMPORARY FILE, THAT WAS CRATD IN THE PREVIOUS STEP>"))
        .shouldDeleteIfEmpty(true)
        .lineAggregator(
                (item) -> item.id() + fieldDelimeter + item.value())
        .append(false)
        .transactional(false)
        .forceSync(true)
        .build()
)

下面是工作要点:

return new JobBuilder("myJob", jobRepository)
    .listener(jobCompletionListener)
    .start(step1)
    .on("COMPLETD").to(step2)
    .on("COMPLETD").to(step3)
    .end()
    .build();

没有人能够就此提供一些见解或建议?

Answer 1

我找到了满足这一要求的替代方法。这里使用的逻辑一

Step 1 will be reading the file and save the processed data in H2 DB
Step 2 will do the same as step 1 using a different processor
In the final step, data will be read from that H2 DB s table and written in the final file
With the above approach, simply use next() to apply sequential flow, instead of conditional flow of from(), to() and on()

因此,狙击手应当看像这种东西。

.start(step1, that will read the file and store in DB)
.next(step2, that will read another file and store in DB)
.next(step3, that will fetch the data from the DB and write in the file)

Note: If the data are directly being written in the file after being fetched from the DB, without any processing, then DON T use any processor, i.e., the step (final one) will be only having a reader (to fetch from the DB) and a writer (to write in the file).

友情链接