Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for reading zero-length values using RawRead #26

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

missinglink
Copy link

@missinglink missinglink commented Nov 4, 2022

When reading zero-length values using RawRead (ie. nil, empty) the getBytes() function triggers a fatal SIGBUS error.
We can avoid this by detecting the case where val.mv_size == 0 and avoiding attempts to access that memory.

resolves #25
ref: https://github.com/ledgerwatch/lmdb-go/blob/master/lmdb/lmdb.h#L255

@AskAlexSharov
Copy link
Contributor

@missinglink - how to separate Nil from empty []byte{} ? Does MDBX support both?

@missinglink
Copy link
Author

LMDB doesn't seem to have a distinction between nil and empty 🤷

@missinglink
Copy link
Author

missinglink commented Nov 4, 2022

The C value val.mv_size is equal to 0 for values stored as either var v_nil []byte or var v_empty = []byte{}, it doesn't provide any way of distinguishing between the two.

@missinglink
Copy link
Author

But reading either form prior to this PR (in combination with RawRead) triggers a panic

@AskAlexSharov
Copy link
Contributor

I think it must be EmptyByteSlice, not nil. Because we using Nil as indicator of “end of iteration over dbi”.

If you insert key of len 0 - it will be sorted in the beginning of dbi, not in the end. So, it behave as empty byte slice inside db, then it must be returned as empty byte slice.

What official C++ bindings return in this case?

@AskAlexSharov
Copy link
Contributor

@missinglink answer from mdbx maintainer https://t.me/libmdbx/3882

@missinglink
Copy link
Author

I think it must be EmptyByteSlice, not nil. Because we using Nil as indicator of “end of iteration over dbi”.

I'm happy to change it to []byte{}, but I'm still not clear if “end of iteration over dbi” is an issue as I believe that the key is checked for nil, not the value and, as far as I can tell, the getBytes() function is only used for values and not keys.

I'll update this PR as I'd like to get something merged which avoids the panic.

Reading zero-length values using `RawRead` (ie. `nil`) the `getBytes()` function triggers a fatal `SIGBUS` error.
We can avoid this by detecting the case where `val.mv_size == 0` and avoiding attempts to access that memory.
@missinglink
Copy link
Author

updated via rebase

@missinglink
Copy link
Author

missinglink commented Nov 7, 2022

Out of curiosity, what does [:val.mv_size:val.mv_size] do exactly? How is it different from [:val.mv_size]?

edit: agh it's called a "Full slice expression" and explicitly sets the capacity too, I learned something new today.

@AskAlexSharov
Copy link
Contributor

I think it must be EmptyByteSlice, not nil. Because we using Nil as indicator of “end of iteration over dbi”.

I'm happy to change it to []byte{}, but I'm still not clear if “end of iteration over dbi” is an issue as I believe that the key is checked for nil, not the value and, as far as I can tell, the getBytes() function is only used for values and not keys.

I'll update this PR as I'd like to get something merged which avoids the panic.

when iterating over DupSort DBI - we using nil-check of value - to indicate end of sub-DBI.

@AskAlexSharov
Copy link
Contributor

@missinglink - just double-confirm. You wan't merge this PR to lmdb-go, not to mdbx-go repo, yes?

@missinglink
Copy link
Author

Correct, for lmdb not mdbx

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bus Error when reading nil values
2 participants