Skip to content

New concept exercise top-secret (concept: ast) #930

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 22 commits into from
Nov 6, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions concepts/ast/.meta/config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"blurb": "The Abstract Syntax Tree is a way to represent Elixir code as data.",
"authors": [
"angelikatyborska"
],
"contributors": [
]
}
43 changes: 43 additions & 0 deletions concepts/ast/about.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# About
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I gave up on writing a custom about.md and just copy-pasted introduction.md because I have doubts about how many students actually read those and I'm low on time.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we will need to circle back to it once we have quote/unquote and macros concepts anyway.


The Abstract Syntax Tree (AST), also called a [_quoted expression_][getting-started-quote], is a way to represent code as data.

Each node in the AST is a three-element tuple.

```elixir
# AST representation of:
# 2 + 3
{:+, [], [2, 3]}
```

The first element, an atom, is the operation. The second element, a keyword list, is the metadata. The third element is a list of arguments, which contains other nodes. Literal values such as integers, atoms, and strings are represented in the AST as themselves instead of three-element tuples.

## Turning code into ASTs

Changing Elixir code to ASTs and ASTs back to code is part of the standard library. You can find functions for working with ASTs in the modules `Code` (e.g. [to change a string with code to an AST][doc-code-string-to-quoted]) and `Macro` (e.g. [to traverse the AST or change it to a string][macro-prewalk]).

Note that all of the functions in the standard library use the name "quoted" to mean the AST (short for _quoted expression_).

The special form for turning code into an AST is called [`quote`][doc-quote]. It accepts a code block and returns its AST.

```elixir
quote do
2 + 3 - 1
end

# => {:-, [], [
# {:+, [], [2, 3]},
# 1
# ]}
```

## Use cases

The ability to represent code as an AST is at the heart of metaprogramming in Elixir. _Macros_, which is a way to write Elixir code that produces Elixir code, work by returning ASTs as output.

Another use case for ASTs is static code analysis, like Exercism's own tool, the Analyzer, which you might already know as the little bot that leaves comments on your solutions.

[getting-started-quote]: https://elixir-lang.org/getting-started/meta/quote-and-unquote.html#quoting
[doc-quote]: https://hexdocs.pm/elixir/Kernel.SpecialForms.html#quote/2
[doc-code-string-to-quoted]: https://hexdocs.pm/elixir/Code.html#string_to_quoted/2
[macro-prewalk]: https://hexdocs.pm/elixir/Macro.html#prewalk/3
38 changes: 38 additions & 0 deletions concepts/ast/introduction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Introduction

The Abstract Syntax Tree (AST), also called a _quoted expression_, is a way to represent code as data.

Each node in the AST is a three-element tuple.

```elixir
# AST representation of:
# 2 + 3
{:+, [], [2, 3]}
```

The first element, an atom, is the operation. The second element, a keyword list, is the metadata. The third element is a list of arguments, which contains other nodes. Literal values such as integers, atoms, and strings are represented in the AST as themselves instead of three-element tuples.

## Turning code into ASTs

Changing Elixir code to ASTs and ASTs back to code is part of the standard library. You can find functions for working with ASTs in the modules `Code` (e.g. to change a string with code to an AST) and `Macro` (e.g. to traverse the AST or change it to a string).

Note that all of the functions in the standard library use the name "quoted" to mean the AST (short for _quoted expression_).

The special form for turning code into an AST is called `quote`. It accepts a code block and returns its AST.

```elixir
quote do
2 + 3 - 1
end

# => {:-, [], [
# {:+, [], [2, 3]},
# 1
# ]}
```

## Use cases

The ability to represent code as an AST is at the heart of metaprogramming in Elixir. _Macros_, which is a way to write Elixir code that produces Elixir code, work by returning ASTs as output.

Another use case for ASTs is static code analysis, like Exercism's own tool, the Analyzer, which you might already know as the little bot that leaves comments on your solutions.
18 changes: 18 additions & 0 deletions concepts/ast/links.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
[
{
"url": "https://elixir-lang.org/getting-started/meta/quote-and-unquote.html#quoting",
"description": "Getting Started - quoting"
},
{
"url": "https://dorgan.ar/posts/2021/04/the_elixir_ast/",
"description": "Introduction to Elixir AST by Lucas San Román"
},
{
"url": "https://hexdocs.pm/elixir/Code.html",
"description": "The Code module"
},
{
"url": "https://hexdocs.pm/elixir/Macro.html",
"description": "The Macro module"
}
]
25 changes: 24 additions & 1 deletion config.json
Original file line number Diff line number Diff line change
Expand Up @@ -596,6 +596,23 @@
"dates-and-time"
],
"status": "beta"
},
{
"slug": "top-secret",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add the prerequisite to dot-dsl too.

"name": "Top Secret",
"uuid": "4caa7349-836b-4d6e-b3cb-b3133c305013",
"concepts": [
"ast"
],
"prerequisites": [
"atoms",
"tuples",
"strings",
"pattern-matching",
"guards",
"enum"
],
"status": "beta"
}
],
"practice": [
Expand Down Expand Up @@ -2647,7 +2664,8 @@
"errors",
"exceptions",
"keyword-lists",
"nil"
"nil",
"ast"
],
"practices": [
"keyword-lists"
Expand Down Expand Up @@ -2874,6 +2892,11 @@
"slug": "anonymous-functions",
"name": "Anonymous Functions"
},
{
"uuid": "550fac1f-c1dc-4eb5-92f2-5ac0e893be2a",
"slug": "ast",
"name": "AST"
},
{
"uuid": "0bc1ad03-5ec7-4269-ad33-df6decbb5ee2",
"slug": "atoms",
Expand Down
51 changes: 51 additions & 0 deletions exercises/concept/top-secret/.docs/hints.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Hints

## General

- Read about quoting in the [official Getting Started guide][getting-started-quote].
- Read the [introduction to Elixir AST by Lucas San Román][ast-intro-lucas].
- Read [the official documentation for `quote`][doc-quote].
- Inspect the output of [`quote`][doc-quote] to familiarize yourself with how ASTs look like for specific code snippets.

## 1. Turn code into data

- There is a [built-in function][doc-code-string-to-quoted] that turns a string with code into an AST.

## 2. Parse a single AST node

- Inspect the output of [`quote`][doc-quote] to familiarize yourself with how ASTs look like for specific code snippets.
- The operations that define a function are `:def` and `:defp`.
- The operation is the first element in a three-element AST node tuple.
- You can ignore the second element in the tuple in this exercise completely.
- The third element in the tuple is the argument list of the operation that defines the function.
- The first element on that list is the function's name, and the second element is the function's body.

## 3. Decode the secret message part from function definition

- Inspect the output of [`quote`][doc-quote] to familiarize yourself with how ASTs look like for specific code snippets.
- The AST node that contains the function's name also contains the function's argument list as the third element.
- The arity of a function is the length of its argument list.
- There is a [built-in function in the `String` module][string-slice] that can get the first `n` characters from a string.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add one more hint:

Suggested change
- There is a [built-in function in the `String` module][string-slice] that can get the first `n` characters from a string.
- There is a [built-in function in the `String` module][string-slice] that can get the first `n` characters from a string.
- A function without arguments written without parentheses will not have a list as argument but an atom.

- A function without arguments written without parentheses will not have a list as argument but an atom.

## 4. Fix the decoding for functions with guards

- Inspect the output of [`quote`][doc-quote] to familiarize yourself with how ASTs look like for specific code snippets.
- When a function has a guard, the third element in the tuple for the `:def/:defp` operation is a bit different.
- That third element is a list with two elements, the first one is the tuple for the `:when` operation, and the second one is the function's body.
- The `:when` operation's arguments are a two-element list, where the first argument is the function's name, and the second is the guard expression.

## 5. Decode the full secret message

- Use the function `to_ast/1` that you implemented in the first task to create the AST.
- There is a [built-in function][macro-prewalk] that can visit each node in an AST with an accumulator.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more:

Suggested change
- There is a [built-in function][macro-prewalk] that can visit each node in an AST with an accumulator.
- There is a [built-in function][macro-prewalk] that can visit each node in an AST with an accumulator.
- Use the function `to_ast/1` that you implemented in the first task to create the AST.

- Use the function `decode_secret_message_part/2` that you implemented in previous tasks to prewalk the AST.
- To reverse the accumulator at the end and turn it into a string, refresh your knowledge of the [`Enum` module][enum].

[getting-started-quote]: https://elixir-lang.org/getting-started/meta/quote-and-unquote.html#quoting
[doc-quote]: https://hexdocs.pm/elixir/Kernel.SpecialForms.html#quote/2
[ast-intro-lucas]: https://dorgan.ar/posts/2021/04/the_elixir_ast/
[doc-code-string-to-quoted]: https://hexdocs.pm/elixir/Code.html#string_to_quoted/2
[string-slice]: https://hexdocs.pm/elixir/String.html#slice/2
[macro-prewalk]: https://hexdocs.pm/elixir/Macro.html#prewalk/3
[enum]: https://hexdocs.pm/elixir/Enum.html
83 changes: 83 additions & 0 deletions exercises/concept/top-secret/.docs/instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Instructions

You're part of a task force fighting against corporate espionage. You have a secret informer at Shady Company X, which you suspect of stealing secrets from its competitors.

Your informer, Agent Ex, is an Elixir developer. She is encoding secret messages in her code.

To decode her secret messages:

- Take all functions (public and private) in the order they're defined in.
- For each function, take the first `n` characters from its name, where `n` is the function's arity.

## 1. Turn code into data

Implement the `TopSecret.to_ast/1` function. It should take a string with Elixir code and return its AST.

```elixir
TopSecret.to_ast("div(4, 3)")
# => {:div, [line: 1], [4, 3]}
```

## 2. Parse a single AST node

Implement the `TopSecret.decode_secret_message_part/2` function. It should take an AST node and an accumulator for the secret message (a list). It should return a tuple with the AST node unchanged as the first element, and the accumulator as the second element.

If the operation of the AST node is defining a function (`def` or `defp`), prepend the function name (changed to a string) to the accumulator. If the operation is something else, return the accumulator unchanged.

```elixir
ast_node = TopSecret.to_ast("defp cat(a, b, c), do: nil")
TopSecret.decode_secret_message_part(ast_node, ["day"])
# => {ast_node, ["cat", "day"]}

ast_node = TopSecret.to_ast("10 + 3")
TopSecret.decode_secret_message_part(ast_node, ["day"])
# => {ast_node, ["day"]}
```

This function doesn't need to do any recursive calls to check the whole AST, only the given node. We will traverse the whole AST with built-in tools in the last step.

## 3. Decode the secret message part from function definition

Extend the `TopSecret.decode_secret_message_part/2` function. If the operation in the AST node is defining a function, don't return the whole function name. Instead, check the function's arity. Then, return only first `n` character from the name, where `n` is the arity.

```elixir
ast_node = TopSecret.to_ast("defp cat(a, b), do: nil")
TopSecret.decode_secret_message_part(ast_node, ["day"])
# => {ast_node, ["ca", "day"]}

ast_node = TopSecret.to_ast("defp cat(), do: nil")
TopSecret.decode_secret_message_part(ast_node, ["day"])
# => {ast_node, ["", "day"]}
```

## 4. Fix the decoding for functions with guards

Extend the `TopSecret.decode_secret_message_part/2` function. Make sure the function's name and arity is correctly detected for function definitions that use guards.

```elixir
ast_node = TopSecret.to_ast("defp cat(a, b) when is_nil(a), do: nil")
TopSecret.decode_secret_message_part(ast_node, ["day"])
# => {ast_node, ["ca", "day"]}
```

## 5. Decode the full secret message

Implement the `TopSecret.decode_secret_message/1` function. It should take a string with Elixir code and return the secret message as a string decoded from all function definitions found in the code. Make sure to reuse functions defined in previous steps.

```elixir
code = """
defmodule MyCalendar do
def busy?(date, time) do
Date.day_of_week(date) != 7 and
time.hour in 10..16
end

def yesterday?(date) do
Date.diff(Date.utc_today, date)
end
end
"""

TopSecret.decode_secret_message(code)
# => "buy"
```
40 changes: 40 additions & 0 deletions exercises/concept/top-secret/.docs/introduction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Introduction

## AST

The Abstract Syntax Tree (AST), also called a _quoted expression_, is a way to represent code as data.

Each node in the AST is a three-element tuple.

```elixir
# AST representation of:
# 2 + 3
{:+, [], [2, 3]}
```

The first element, an atom, is the operation. The second element, a keyword list, is the metadata. The third element is a list of arguments, which contains other nodes. Literal values such as integers, atoms, and strings are represented in the AST as themselves instead of three-element tuples.

### Turning code into ASTs

Changing Elixir code to ASTs and ASTs back to code is part of the standard library. You can find functions for working with ASTs in the modules `Code` (e.g. to change a string with code to an AST) and `Macro` (e.g. to traverse the AST or change it to a string).

Note that all of the functions in the standard library use the name "quoted" to mean the AST (short for _quoted expression_).

The special form for turning code into an AST is called `quote`. It accepts a code block and returns its AST.

```elixir
quote do
2 + 3 - 1
end

# => {:-, [], [
# {:+, [], [2, 3]},
# 1
# ]}
```

### Use cases

The ability to represent code as an AST is at the heart of metaprogramming in Elixir. _Macros_, which is a way to write Elixir code that produces Elixir code, work by returning ASTs as output.

Another use case for ASTs is static code analysis, like Exercism's own tool, the Analyzer, which you might already know as the little bot that leaves comments on your solutions.
4 changes: 4 additions & 0 deletions exercises/concept/top-secret/.formatter.exs
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Used by "mix format"
[
inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"]
]
24 changes: 24 additions & 0 deletions exercises/concept/top-secret/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# The directory Mix will write compiled artifacts to.
/_build/

# If you run "mix test --cover", coverage assets end up here.
/cover/

# The directory Mix downloads your dependencies sources to.
/deps/

# Where third-party dependencies like ExDoc output generated docs.
/doc/

# Ignore .fetch files in case you like to edit your project deps locally.
/.fetch

# If the VM crashes, it generates a dump, let's ignore it too.
erl_crash.dump

# Also ignore archive artifacts (built via "mix archive.build").
*.ez

# Ignore package tarball (built via "mix hex.build").
maps-*.tar

19 changes: 19 additions & 0 deletions exercises/concept/top-secret/.meta/config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"blurb": "Learn about the Abstract Syntax Tree (AST) by helping decode secret messages from Agent Ex.",
"authors": [
"jiegillet",
"angelikatyborska"
],
"files": {
"solution": [
"lib/top_secret.ex"
],
"test": [
"test/top_secret_test.exs"
],
"exemplar": [
".meta/exemplar.ex"
]
},
"language_versions": ">=1.10"
}
Loading