This repository was archived by the owner on Sep 1, 2022. It is now read-only.
File decompression uses new class StringKeeper #1153
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
File decompression uses new class StringKeeper to make sure a file is being handled only
by one thread at a time. StringKeeper maintains a synchronised list of strings (in this case file paths) across threads. When a thread wishes to access a file it can check the list to see if the file is already in use and thread wait() until a notifyAll() is received. notifyAll() is sent when a thread finishes with a file.
This addresses the following issue:
While unzipping files other concurrent requests waiting for the file are unblocked prior to the file being unzipped causing issues trying to read half unzipped files.
The cause of the problem is a race condition between the caching system (FileCache.java) and the decompression in NetcdfFile.java. NetcdfFile.makeUncompressed() adequetely locks files across threads (by catching OverlappingFileLockException) to ensure mutiple threads do not attempt to decompress a file concurrently. However other threads trying to read from files (eg N3header.isValidFile, which results in the "not a valid CDM file" error) will not be aware that a file is still be written and end up reading wrong or no bytes. This is because the FileCache is not checking the file locks established during decompression.