Merge 4-0-254 to backup_sync #181
Open
+38,927
−36,340
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
* fix assignmentModulo
* fix assignmentXor
* fix assignmentOr
* fix assignmentAnd
* fix assignmentExponentiation
* fix assignmentShiftLeft
* fix assignment{Logical,Arithmetic}ShiftRight
* scalafmt
…#4954) These synthetic METHOD nodes did not have an offset at all. For: https://shiftleftinc.atlassian.net/browse/SEN-3250
…o#4955) As soon as `FullNameSemantics` started logging duplicate semantics (in joernio#4954), it was observed that `SemanticTestCpg` was passing a fresh Semantics instance to `EngineContext` instead of passing the one used by `OssDataFlow`. By coincidence, however, they were the same. Nevertheless, this patch guarantees they are the same.
Using the CDT API (getContainingFilename) now. For: joernio#4924
* [ruby] ANTLR Profiler Summary * Introduced global profiling to summarize rule and parse performance across the project * Added a shutdown hook to dump a summary of the profiler rules in a file `antlr_summary.log` at the root of the project with this information * Check parent exists before dumping. May not be there during test cases * Added note around shutdown hook
Sometimes AST trees would be thrown away, specifically when long chained expressions would be compressed into temporary cache variables. This only happens with expressions, and closures typically write directly to the diff graph. This PR minimizes what is written directly to the diff graph from function creation, and extends `x2cpg.Ast` to also include `CAPTURE` edges. This also introduces `closureToRefs` map to track which `Block` nodes already have live method ASTs in the diff graph.
…rnio#4983) This reverts commit 3c27bf6.
* [ruby] Changed handling of ForEach loops * [ruby] fixed double _astIn reference to identifier * [ruby] fix failing tests
…" (joernio#4985) * Revert "Revert "[ruby] Ignore "Throwaway" AST Structures (joernio#4982)" (joernio#4983)" This reverts commit 464480d. * [ruby] Re-implemented "Ignore "Throwaway" AST Structures (joernio#4982)" This correctly prevents re-use of nodes that are already being used elsewhere by ensuring deep copies.
`RubyExpression` nodes don't inherently consider the `span` in the calculation of its hash, so when put into any hashed context, nodes that only rely on `span` alone will collide in these contexts.
… dir (joernio#4956) The logic to guess the base dir of the installation is quite fiddly but works for our use cases for astgen. PhpParser implemented something similar, but not quite - and it failed for buildbot. On buildbot the installation path for php2cpg is `/worker/sptestV2-php2cpg/build/x2cpg-internal/php2cpg/target/universal/stage` which (prior to this PR) leads to an invalid derived php-parser name and the following error: ``` 2024-09-25 09:30:08.623 ERROR Invalid path for PhpParserBin: /worker/sptestV2-/php2cpg/bin/php-parser/php-parser.php ```
…nse header (joernio#4995) Turns out the HTTPServer expects an explicit "Connection: close" header which requests the connection to be closed after the transaction ends. Otherwise, it would wait for a 10sec timeout for the next thread to become available. In case we only allow for one thread (jimple2cpg) that would mean additional waiting which renders the whole server approach useless. This in now fixes as we immediately close the connection after the frontend is finished.
This happened for parameter and variable declarations in parentheses.
…ex for named arguments (joernio#5000)
* upgrade deps * trigger ci
…ed arguments (joernio#5003) * [dataflowengineoss] Fix PassThroughMapping criteria for named arguments * sort result for testing purposes
…o#5273) GitHub code scanning has strict requirements on SARIF files that need to be enforced, such as minimum elements in an array, line number > 0, etc. Some validators emit warnings on this, but GH fails the pipeline. Additionally, GitHub makes use of fingerprinting to avoid duplication between versions. The SARIF conversion has been adapted to accommodate the above.
… file signature format (joernio#5274) * First version of a JVM binary signature calculator for java parser classes. * Add binary signatures to javasrc nodes * Add generic signature for native foreach iterator tmp local * Do some cleanup * Add test for type parameter with multiple interface bounds * Add generic tests description and add unspecified type to lambda type decls --------- Co-authored-by: Markus Lottmann <markus.lottmann@mlbox.net>
* [pysrc2cpg] Refactor convert methods for function lowering. Use shared implementation. * Separate method full names for redefined functions. * Fmt
…151) (joernio#5286) * handle errors gracefully * use scala best practices * trim dependency names and versions * test for asserting if version is trimmed Co-authored-by: Karan Batavia <118820668+karan-batavia@users.noreply.github.com>
…ls (joernio#5290) * [c#] initial support for replacing setter assignments with setter calls * translateBinaryOperatorName -> binaryOperatorsMap
* Using Block Scope fix (#109) * fix methodFullName inside using block * code refactoring * Try handling change * added curly braces * fix import * review comments --------- Co-authored-by: Ankit Kumar <118803988+ankit-privado@users.noreply.github.com>
* fix - partial ast gen result in case of failure --------- Co-authored-by: PallaviShreshtha <120652805+PallaviShreshtha@users.noreply.github.com>
In the case where an unexpected string in placed in the `filename` property, default to attempting URI construction within a `Try` and returning a `None` on failure
* [c#] fix synthetic set_* method call signature * [c#] support setter assignments via += et al assignments
Lowering regex matches to reflect assignments made to global variables. The lowering becomes: ```ruby tmp = 'hello'.match(/h(el)lo/) if tmp $~ = tmp $& = tmp[0] tmp.begin(0) else $~ = nil $& = nil nil end ``` Some heuristics to assign group matching global variables is next.
This PR implements the other component of regex matching defining global variables. In Ruby, `$1`, `$2`, etc. correspond to the group matched in the last match. This is synonymous to how a `MatchData` object could refer to these matches. This PR models these `nref` objects to `$[1]` and, during a match lowering, defines them to the corresponding index position of the lowered temp match object, i.e., `$[1] = <tmp-0>[1]` where `N` is determined by the number of opening parenthesis (simple heuristic). Additionally, the lowered `match` calls have their `methodFullName` defined for convenient policy/semantic definition creation.
It was recently determined that bundled types were prefixed with the `__core.Kernel` prefix which was unintended, as it should only go as far as `__core`. This change remediates this, as well as logs a warning when a full name is attempted to be constructed in a way that will generate a bundled type with the `__core.Kernel` prefix, or if a non-bundled type will be created with a `__core` prefix.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Created by Github action