feat: better JSON parsing

[klkvr]

Component

Forge

Describe the feature you would like

Current UX for JSON-parsing of objects is not great. Users are required to place JSON fields in alphabetical order, and parseJson basically guesses types, making it impossible to parse e.g. fixed-length arrays or bytes of length 20 or 32.

To fix this we need to make JSON parser aware of struct field types and names. This issue proposes the following approach.

1. Add new cheatcodes

   function parseJsonStruct(string calldata json, string calldata schema) external pure returns (bytes memory abiEncodedData);
   
   function parseJsonStruct(string calldata json, string calldata key, string calldata schema) external pure returns (bytes memory abiEncodedData);

   Those cheatcodes will be identical to parseJson, but will be guided by types from schema. Format for schema would be the json representation of Vec<StructField> where StructField is:

   struct StructField {
        /// Name of the field which will be used when parsing JSON.
        name: String,
        /// Type of the field
        ty: StructFieldType,
   }
   
   /// Solidity type representation which can be (de)serialized from Json and converted into DynSolType
   enum StructFieldType {
       Struct(Vec<StructField>),
       Array(Box<StructFieldType>),
       FixedArray(Box<StructFieldType>, usize),
       /// Inner value must be decodable into DynSolType.
       Primitive(String),
   }

   Another way to represent this in JSON could be by using alloy_dyn_abi::Resolver used for EIP-712 + type name.

2. Add helpers for generating schema values. For example, we could add forge bind-json command which would accept a path to .sol file, and produce either a schema for all structs, or a comlete parsing library looking like:

library Helpers {
    string constant SCHEMA_MyStruct = '[{"name": "field1", "ty": "uint256"}, ...]';
    string constant SCHEMA_AnotherStruct = ...;

    function parseMyStruct(string memory json) internal pure returns(MyStruct memory) {
        return abi.decode(vm.parseJsonStruct(json, SCHEMA_MyStruct), (MyStruct));
    }
    ...
}

The same approach can be used for serialization of structs as well:

function serializeStruct(string calldata schema, bytes memory abiEncodedData)
        external
        returns (string memory json);

function serializeStruct(string calldata objectKey, string calldata valueKey, string calldata schema, bytes memory abiEncodedData)
        external
        returns (string memory json);

Such approach reduces compilation overhead by keeping most of the logic in cheatcode implementation, and only requiring contracts to only contain relatively small schema definitions.

[zerosnacks]

This would be great, I've been getting quite a few questions around JSON parsing / writing and the current API for serialization / deserialization is difficult to work with

[mattsse]

because there's no way to get the abi of a tuple at runtime we're forced to pass this as args, we also need the fields to get around the ordering problem.

this approach is doable, we just need to make it easy to get the schema of a type, I think this can even be combined with loading the schema from a json file itself and forge could generate those

[klkvr]

I think ideally we can do custom preprocessing here with custom cache. e.g. store a mapping (struct name -> struct schema) which is derived from AST on each non-cached compiler run and kept along with artifacts. it can be cheap to generate if we'll do it in parallel with running the solc, and would allow users to just do parseJsonStruct(json, "StructName") instead of manually updating schema each time when new fields are added.