Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Object api strings return bytes instead of str [Python] #5997

Closed
W1M0R opened this issue Jun 23, 2020 · 2 comments · May be fixed by #8551
Closed

Object api strings return bytes instead of str [Python] #5997

W1M0R opened this issue Jun 23, 2020 · 2 comments · May be fixed by #8551

Comments

@W1M0R
Copy link

W1M0R commented Jun 23, 2020

Given the following generated Python object api class:

class LoginInputT(object):

    # LoginInputT
    def __init__(self):
        self.username = None  # type: str
        self.password = None  # type: str

    @classmethod
    def InitFromBuf(cls, buf, pos):
        loginInput = LoginInput()
        loginInput.Init(buf, pos)
        return cls.InitFromObj(loginInput)

    @classmethod
    def InitFromObj(cls, loginInput):
        x = LoginInputT()
        x._UnPack(loginInput)
        return x

    # LoginInputT
    def _UnPack(self, loginInput):
        if loginInput is None:
            return
        self.username = loginInput.Username()
        self.password = loginInput.Password()

    # LoginInputT
    def Pack(self, builder):
        if self.username is not None:
            username = builder.CreateString(self.username)
        if self.password is not None:
            password = builder.CreateString(self.password)
        LoginInputStart(builder)
        if self.username is not None:
            LoginInputAddUsername(builder, username)
        if self.password is not None:
            LoginInputAddPassword(builder, password)
        loginInput = LoginInputEnd(builder)
        return loginInput

And schema:

table LoginInput {
  username:string;
  password:string;
}

I would expect that self.username and self.password would be of type str after a call to InitFromObj, but instead it is of type bytes. Examples in the documentation indicate that CreateString always accept a str parameter, so if the Pack function does builder.CreateString(self.username) then the opposite action in UnPack should result in str type for self.username.

Since flatbuffers schema supports string, [byte], and [ubyte] fields, I would expect that the generated Python code also honours that distinction. A library such as jsons (that generates JSON from any Python object) can not know that some fields that have type bytes should be interpreted as strings and others of type bytes are actually bytes.

The following code snippet using the jsons package, allows me to generate JSON data with the expected output types:

import os
import sys
import jsons
import flatbuffers
import MyTest.MyTest
myTest= MyTest.MyTest.MyTest.GetRootAsMyTest(buf, offs)
myTestT= MyTest.MyTest.MyTestT.InitFromObj(myTest)
def bytes_serializer(obj: bytes, detect_str: bool = False, **kwargs):
  if detect_str:
    try:
      return obj.decode('utf-8')
    except UnicodeError:
      return obj
  return obj
jsons.set_serializer(bytes_serializer, bytes)
return jsons.dumps(myTestT, key_transformer=jsons.KEY_TRANSFORMER_SNAKECASE, strip_privates=True, strip_nulls=True, strip_class_variables=True, strict=True, detect_str=True)
@aardappel
Copy link
Collaborator

I am guessing bytes is more efficient since there's no decoding required?
@lu-wang-g

@github-actions
Copy link
Contributor

This issue is stale because it has been open 6 months with no activity. Please comment or this will be closed in 14 days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants