-
-
Notifications
You must be signed in to change notification settings - Fork 403
New concept exercise top-secret
(concept: ast
)
#930
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
ec3fabe
8285581
68fa46b
4365437
b1e0b2d
85c8789
0990439
88fa3e1
3f0f8f4
1150d07
46c2af7
f6eef8d
9e50fb7
bc72872
4582a74
b6fbbfd
862c8dd
42b6cc7
0840b8b
559dcd1
7ff4437
e6ec12f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
{ | ||
"blurb": "The Abstract Syntax Tree is a way to represent Elixir code as data.", | ||
"authors": [ | ||
"angelikatyborska" | ||
], | ||
"contributors": [ | ||
] | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
# About | ||
|
||
The Abstract Syntax Tree (AST), also called a [_quoted expression_][getting-started-quote], is a way to represent code as data. | ||
|
||
Each node in the AST is a three-element tuple. | ||
|
||
```elixir | ||
# AST representation of: | ||
# 2 + 3 | ||
{:+, [], [2, 3]} | ||
``` | ||
|
||
The first element, an atom, is the operation. The second element, a keyword list, is the metadata. The third element is a list of arguments, which contains other nodes. Literal values such as integers, atoms, and strings are represented in the AST as themselves instead of three-element tuples. | ||
|
||
## Turning code into ASTs | ||
|
||
Changing Elixir code to ASTs and ASTs back to code is part of the standard library. You can find functions for working with ASTs in the modules `Code` (e.g. [to change a string with code to an AST][doc-code-string-to-quoted]) and `Macro` (e.g. [to traverse the AST or change it to a string][macro-prewalk]). | ||
|
||
Note that all of the functions in the standard library use the name "quoted" to mean the AST (short for _quoted expression_). | ||
|
||
The special form for turning code into an AST is called [`quote`][doc-quote]. It accepts a code block and returns its AST. | ||
|
||
```elixir | ||
quote do | ||
2 + 3 - 1 | ||
end | ||
|
||
# => {:-, [], [ | ||
# {:+, [], [2, 3]}, | ||
# 1 | ||
# ]} | ||
``` | ||
|
||
## Use cases | ||
|
||
The ability to represent code as an AST is at the heart of metaprogramming in Elixir. _Macros_, which is a way to write Elixir code that produces Elixir code, work by returning ASTs as output. | ||
|
||
Another use case for ASTs is static code analysis, like Exercism's own tool, the Analyzer, which you might already know as the little bot that leaves comments on your solutions. | ||
|
||
[getting-started-quote]: https://elixir-lang.org/getting-started/meta/quote-and-unquote.html#quoting | ||
[doc-quote]: https://hexdocs.pm/elixir/Kernel.SpecialForms.html#quote/2 | ||
[doc-code-string-to-quoted]: https://hexdocs.pm/elixir/Code.html#string_to_quoted/2 | ||
[macro-prewalk]: https://hexdocs.pm/elixir/Macro.html#prewalk/3 |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
# Introduction | ||
|
||
The Abstract Syntax Tree (AST), also called a _quoted expression_, is a way to represent code as data. | ||
|
||
Each node in the AST is a three-element tuple. | ||
|
||
```elixir | ||
# AST representation of: | ||
# 2 + 3 | ||
{:+, [], [2, 3]} | ||
``` | ||
|
||
The first element, an atom, is the operation. The second element, a keyword list, is the metadata. The third element is a list of arguments, which contains other nodes. Literal values such as integers, atoms, and strings are represented in the AST as themselves instead of three-element tuples. | ||
|
||
## Turning code into ASTs | ||
|
||
Changing Elixir code to ASTs and ASTs back to code is part of the standard library. You can find functions for working with ASTs in the modules `Code` (e.g. to change a string with code to an AST) and `Macro` (e.g. to traverse the AST or change it to a string). | ||
|
||
Note that all of the functions in the standard library use the name "quoted" to mean the AST (short for _quoted expression_). | ||
|
||
The special form for turning code into an AST is called `quote`. It accepts a code block and returns its AST. | ||
|
||
```elixir | ||
quote do | ||
2 + 3 - 1 | ||
end | ||
|
||
# => {:-, [], [ | ||
# {:+, [], [2, 3]}, | ||
# 1 | ||
# ]} | ||
``` | ||
|
||
## Use cases | ||
|
||
The ability to represent code as an AST is at the heart of metaprogramming in Elixir. _Macros_, which is a way to write Elixir code that produces Elixir code, work by returning ASTs as output. | ||
|
||
Another use case for ASTs is static code analysis, like Exercism's own tool, the Analyzer, which you might already know as the little bot that leaves comments on your solutions. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
[ | ||
{ | ||
"url": "https://elixir-lang.org/getting-started/meta/quote-and-unquote.html#quoting", | ||
"description": "Getting Started - quoting" | ||
}, | ||
{ | ||
"url": "https://dorgan.ar/posts/2021/04/the_elixir_ast/", | ||
"description": "Introduction to Elixir AST by Lucas San Román" | ||
}, | ||
{ | ||
"url": "https://hexdocs.pm/elixir/Code.html", | ||
"description": "The Code module" | ||
}, | ||
{ | ||
"url": "https://hexdocs.pm/elixir/Macro.html", | ||
"description": "The Macro module" | ||
} | ||
] |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -596,6 +596,23 @@ | |
"dates-and-time" | ||
], | ||
"status": "beta" | ||
}, | ||
{ | ||
"slug": "top-secret", | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should add the prerequisite to |
||
"name": "Top Secret", | ||
"uuid": "4caa7349-836b-4d6e-b3cb-b3133c305013", | ||
"concepts": [ | ||
"ast" | ||
], | ||
"prerequisites": [ | ||
"atoms", | ||
"tuples", | ||
"strings", | ||
"pattern-matching", | ||
"guards", | ||
"enum" | ||
], | ||
"status": "beta" | ||
} | ||
], | ||
"practice": [ | ||
|
@@ -2647,7 +2664,8 @@ | |
"errors", | ||
"exceptions", | ||
"keyword-lists", | ||
"nil" | ||
"nil", | ||
"ast" | ||
], | ||
"practices": [ | ||
"keyword-lists" | ||
|
@@ -2874,6 +2892,11 @@ | |
"slug": "anonymous-functions", | ||
"name": "Anonymous Functions" | ||
}, | ||
{ | ||
"uuid": "550fac1f-c1dc-4eb5-92f2-5ac0e893be2a", | ||
"slug": "ast", | ||
"name": "AST" | ||
}, | ||
{ | ||
"uuid": "0bc1ad03-5ec7-4269-ad33-df6decbb5ee2", | ||
"slug": "atoms", | ||
|
Original file line number | Diff line number | Diff line change | ||||||
---|---|---|---|---|---|---|---|---|
@@ -0,0 +1,51 @@ | ||||||||
# Hints | ||||||||
|
||||||||
## General | ||||||||
|
||||||||
- Read about quoting in the [official Getting Started guide][getting-started-quote]. | ||||||||
- Read the [introduction to Elixir AST by Lucas San Román][ast-intro-lucas]. | ||||||||
- Read [the official documentation for `quote`][doc-quote]. | ||||||||
- Inspect the output of [`quote`][doc-quote] to familiarize yourself with how ASTs look like for specific code snippets. | ||||||||
|
||||||||
## 1. Turn code into data | ||||||||
|
||||||||
- There is a [built-in function][doc-code-string-to-quoted] that turns a string with code into an AST. | ||||||||
|
||||||||
## 2. Parse a single AST node | ||||||||
|
||||||||
- Inspect the output of [`quote`][doc-quote] to familiarize yourself with how ASTs look like for specific code snippets. | ||||||||
- The operations that define a function are `:def` and `:defp`. | ||||||||
- The operation is the first element in a three-element AST node tuple. | ||||||||
- You can ignore the second element in the tuple in this exercise completely. | ||||||||
- The third element in the tuple is the argument list of the operation that defines the function. | ||||||||
- The first element on that list is the function's name, and the second element is the function's body. | ||||||||
|
||||||||
## 3. Decode the secret message part from function definition | ||||||||
|
||||||||
- Inspect the output of [`quote`][doc-quote] to familiarize yourself with how ASTs look like for specific code snippets. | ||||||||
- The AST node that contains the function's name also contains the function's argument list as the third element. | ||||||||
- The arity of a function is the length of its argument list. | ||||||||
- There is a [built-in function in the `String` module][string-slice] that can get the first `n` characters from a string. | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would add one more hint:
Suggested change
|
||||||||
- A function without arguments written without parentheses will not have a list as argument but an atom. | ||||||||
|
||||||||
## 4. Fix the decoding for functions with guards | ||||||||
|
||||||||
- Inspect the output of [`quote`][doc-quote] to familiarize yourself with how ASTs look like for specific code snippets. | ||||||||
- When a function has a guard, the third element in the tuple for the `:def/:defp` operation is a bit different. | ||||||||
- That third element is a list with two elements, the first one is the tuple for the `:when` operation, and the second one is the function's body. | ||||||||
- The `:when` operation's arguments are a two-element list, where the first argument is the function's name, and the second is the guard expression. | ||||||||
|
||||||||
## 5. Decode the full secret message | ||||||||
|
||||||||
- Use the function `to_ast/1` that you implemented in the first task to create the AST. | ||||||||
- There is a [built-in function][macro-prewalk] that can visit each node in an AST with an accumulator. | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. One more:
Suggested change
|
||||||||
- Use the function `decode_secret_message_part/2` that you implemented in previous tasks to prewalk the AST. | ||||||||
- To reverse the accumulator at the end and turn it into a string, refresh your knowledge of the [`Enum` module][enum]. | ||||||||
|
||||||||
[getting-started-quote]: https://elixir-lang.org/getting-started/meta/quote-and-unquote.html#quoting | ||||||||
[doc-quote]: https://hexdocs.pm/elixir/Kernel.SpecialForms.html#quote/2 | ||||||||
[ast-intro-lucas]: https://dorgan.ar/posts/2021/04/the_elixir_ast/ | ||||||||
[doc-code-string-to-quoted]: https://hexdocs.pm/elixir/Code.html#string_to_quoted/2 | ||||||||
[string-slice]: https://hexdocs.pm/elixir/String.html#slice/2 | ||||||||
[macro-prewalk]: https://hexdocs.pm/elixir/Macro.html#prewalk/3 | ||||||||
[enum]: https://hexdocs.pm/elixir/Enum.html |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,83 @@ | ||
# Instructions | ||
|
||
You're part of a task force fighting against corporate espionage. You have a secret informer at Shady Company X, which you suspect of stealing secrets from its competitors. | ||
angelikatyborska marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Your informer, Agent Ex, is an Elixir developer. She is encoding secret messages in her code. | ||
angelikatyborska marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
To decode her secret messages: | ||
|
||
- Take all functions (public and private) in the order they're defined in. | ||
- For each function, take the first `n` characters from its name, where `n` is the function's arity. | ||
|
||
## 1. Turn code into data | ||
|
||
Implement the `TopSecret.to_ast/1` function. It should take a string with Elixir code and return its AST. | ||
|
||
```elixir | ||
TopSecret.to_ast("div(4, 3)") | ||
# => {:div, [line: 1], [4, 3]} | ||
``` | ||
|
||
## 2. Parse a single AST node | ||
|
||
Implement the `TopSecret.decode_secret_message_part/2` function. It should take an AST node and an accumulator for the secret message (a list). It should return a tuple with the AST node unchanged as the first element, and the accumulator as the second element. | ||
|
||
If the operation of the AST node is defining a function (`def` or `defp`), prepend the function name (changed to a string) to the accumulator. If the operation is something else, return the accumulator unchanged. | ||
|
||
```elixir | ||
ast_node = TopSecret.to_ast("defp cat(a, b, c), do: nil") | ||
TopSecret.decode_secret_message_part(ast_node, ["day"]) | ||
# => {ast_node, ["cat", "day"]} | ||
|
||
ast_node = TopSecret.to_ast("10 + 3") | ||
TopSecret.decode_secret_message_part(ast_node, ["day"]) | ||
# => {ast_node, ["day"]} | ||
``` | ||
|
||
This function doesn't need to do any recursive calls to check the whole AST, only the given node. We will traverse the whole AST with built-in tools in the last step. | ||
|
||
## 3. Decode the secret message part from function definition | ||
|
||
Extend the `TopSecret.decode_secret_message_part/2` function. If the operation in the AST node is defining a function, don't return the whole function name. Instead, check the function's arity. Then, return only first `n` character from the name, where `n` is the arity. | ||
|
||
```elixir | ||
ast_node = TopSecret.to_ast("defp cat(a, b), do: nil") | ||
TopSecret.decode_secret_message_part(ast_node, ["day"]) | ||
# => {ast_node, ["ca", "day"]} | ||
|
||
ast_node = TopSecret.to_ast("defp cat(), do: nil") | ||
TopSecret.decode_secret_message_part(ast_node, ["day"]) | ||
# => {ast_node, ["", "day"]} | ||
``` | ||
|
||
## 4. Fix the decoding for functions with guards | ||
|
||
Extend the `TopSecret.decode_secret_message_part/2` function. Make sure the function's name and arity is correctly detected for function definitions that use guards. | ||
|
||
```elixir | ||
ast_node = TopSecret.to_ast("defp cat(a, b) when is_nil(a), do: nil") | ||
TopSecret.decode_secret_message_part(ast_node, ["day"]) | ||
# => {ast_node, ["ca", "day"]} | ||
``` | ||
|
||
## 5. Decode the full secret message | ||
|
||
Implement the `TopSecret.decode_secret_message/1` function. It should take a string with Elixir code and return the secret message as a string decoded from all function definitions found in the code. Make sure to reuse functions defined in previous steps. | ||
|
||
```elixir | ||
code = """ | ||
defmodule MyCalendar do | ||
def busy?(date, time) do | ||
Date.day_of_week(date) != 7 and | ||
time.hour in 10..16 | ||
end | ||
|
||
def yesterday?(date) do | ||
Date.diff(Date.utc_today, date) | ||
end | ||
end | ||
""" | ||
|
||
TopSecret.decode_secret_message(code) | ||
# => "buy" | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
# Introduction | ||
|
||
## AST | ||
|
||
The Abstract Syntax Tree (AST), also called a _quoted expression_, is a way to represent code as data. | ||
|
||
Each node in the AST is a three-element tuple. | ||
|
||
```elixir | ||
# AST representation of: | ||
# 2 + 3 | ||
{:+, [], [2, 3]} | ||
``` | ||
|
||
The first element, an atom, is the operation. The second element, a keyword list, is the metadata. The third element is a list of arguments, which contains other nodes. Literal values such as integers, atoms, and strings are represented in the AST as themselves instead of three-element tuples. | ||
|
||
### Turning code into ASTs | ||
|
||
Changing Elixir code to ASTs and ASTs back to code is part of the standard library. You can find functions for working with ASTs in the modules `Code` (e.g. to change a string with code to an AST) and `Macro` (e.g. to traverse the AST or change it to a string). | ||
|
||
Note that all of the functions in the standard library use the name "quoted" to mean the AST (short for _quoted expression_). | ||
|
||
The special form for turning code into an AST is called `quote`. It accepts a code block and returns its AST. | ||
|
||
```elixir | ||
quote do | ||
2 + 3 - 1 | ||
end | ||
|
||
# => {:-, [], [ | ||
# {:+, [], [2, 3]}, | ||
# 1 | ||
# ]} | ||
``` | ||
|
||
### Use cases | ||
|
||
The ability to represent code as an AST is at the heart of metaprogramming in Elixir. _Macros_, which is a way to write Elixir code that produces Elixir code, work by returning ASTs as output. | ||
|
||
Another use case for ASTs is static code analysis, like Exercism's own tool, the Analyzer, which you might already know as the little bot that leaves comments on your solutions. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
# Used by "mix format" | ||
[ | ||
inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"] | ||
] |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
# The directory Mix will write compiled artifacts to. | ||
/_build/ | ||
|
||
# If you run "mix test --cover", coverage assets end up here. | ||
/cover/ | ||
|
||
# The directory Mix downloads your dependencies sources to. | ||
/deps/ | ||
|
||
# Where third-party dependencies like ExDoc output generated docs. | ||
/doc/ | ||
|
||
# Ignore .fetch files in case you like to edit your project deps locally. | ||
/.fetch | ||
|
||
# If the VM crashes, it generates a dump, let's ignore it too. | ||
erl_crash.dump | ||
|
||
# Also ignore archive artifacts (built via "mix archive.build"). | ||
*.ez | ||
|
||
# Ignore package tarball (built via "mix hex.build"). | ||
maps-*.tar | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
{ | ||
"blurb": "Learn about the Abstract Syntax Tree (AST) by helping decode secret messages from Agent Ex.", | ||
"authors": [ | ||
"jiegillet", | ||
"angelikatyborska" | ||
], | ||
"files": { | ||
"solution": [ | ||
"lib/top_secret.ex" | ||
], | ||
"test": [ | ||
"test/top_secret_test.exs" | ||
], | ||
"exemplar": [ | ||
".meta/exemplar.ex" | ||
] | ||
}, | ||
"language_versions": ">=1.10" | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I gave up on writing a custom
about.md
and just copy-pastedintroduction.md
because I have doubts about how many students actually read those and I'm low on time.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we will need to circle back to it once we have quote/unquote and macros concepts anyway.