Skip to content
This repository was archived by the owner on Sep 1, 2022. It is now read-only.

File decompression uses new class StringKeeper #1153

Merged
merged 1 commit into from
Sep 24, 2018

Conversation

craigrose
Copy link
Contributor

File decompression uses new class StringKeeper to make sure a file is being handled only
by one thread at a time. StringKeeper maintains a synchronised list of strings (in this case file paths) across threads. When a thread wishes to access a file it can check the list to see if the file is already in use and thread wait() until a notifyAll() is received. notifyAll() is sent when a thread finishes with a file.

This addresses the following issue:

While unzipping files other concurrent requests waiting for the file are unblocked prior to the file being unzipped causing issues trying to read half unzipped files.

The cause of the problem is a race condition between the caching system (FileCache.java) and the decompression in NetcdfFile.java. NetcdfFile.makeUncompressed() adequetely locks files across threads (by catching OverlappingFileLockException) to ensure mutiple threads do not attempt to decompress a file concurrently. However other threads trying to read from files (eg N3header.isValidFile, which results in the "not a valid CDM file" error) will not be aware that a file is still be written and end up reading wrong or no bytes. This is because the FileCache is not checking the file locks established during decompression.

#22)

File decompression uses new class StringLocker to make sure a file is being handled only
by one thread at a time
@CLAassistant
Copy link

CLA assistant check
Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@craigrose craigrose changed the title File decompression uses new class StringKeeper to avoid File decompression uses new class StringKeeper Sep 13, 2018
@lesserwhirls
Copy link
Collaborator

Thank you again, and sorry for the delay in getting this merged in!

@lesserwhirls lesserwhirls merged commit 8cc1308 into Unidata:master Sep 24, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants