370 lines
59 KiB
HTML
370 lines
59 KiB
HTML
<!DOCTYPE html>
|
||
<html lang="en"><head><meta charset="UTF-8"/><meta name="viewport" content="width=device-width, initial-scale=1.0"/><title>Metaprogramming · The Julia Language</title><script>(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
|
||
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
|
||
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
|
||
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
|
||
|
||
ga('create', 'UA-28835595-6', 'auto');
|
||
ga('send', 'pageview');
|
||
</script><link href="https://cdnjs.cloudflare.com/ajax/libs/normalize/4.2.0/normalize.min.css" rel="stylesheet" type="text/css"/><link href="https://fonts.googleapis.com/css?family=Lato|Roboto+Mono" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.6.3/css/font-awesome.min.css" rel="stylesheet" type="text/css"/><script>documenterBaseURL=".."</script><script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.2.0/require.min.js" data-main="../assets/documenter.js"></script><script src="../siteinfo.js"></script><script src="../../versions.js"></script><link href="../assets/highlightjs/default.css" rel="stylesheet" type="text/css"/><link href="../assets/documenter.css" rel="stylesheet" type="text/css"/></head><body><nav class="toc"><a href="../index.html"><img class="logo" src="../assets/logo.png" alt="The Julia Language logo"/></a><h1>The Julia Language</h1><select id="version-selector" onChange="window.location.href=this.value" style="visibility: hidden"></select><form class="search" action="../search.html"><input id="search-query" name="q" type="text" placeholder="Search docs"/></form><ul><li><a class="toctext" href="../index.html">Home</a></li><li><span class="toctext">Manual</span><ul><li><a class="toctext" href="introduction.html">Introduction</a></li><li><a class="toctext" href="getting-started.html">Getting Started</a></li><li><a class="toctext" href="variables.html">Variables</a></li><li><a class="toctext" href="integers-and-floating-point-numbers.html">Integers and Floating-Point Numbers</a></li><li><a class="toctext" href="mathematical-operations.html">Mathematical Operations and Elementary Functions</a></li><li><a class="toctext" href="complex-and-rational-numbers.html">Complex and Rational Numbers</a></li><li><a class="toctext" href="strings.html">Strings</a></li><li><a class="toctext" href="functions.html">Functions</a></li><li><a class="toctext" href="control-flow.html">Control Flow</a></li><li><a class="toctext" href="variables-and-scoping.html">Scope of Variables</a></li><li><a class="toctext" href="types.html">Types</a></li><li><a class="toctext" href="methods.html">Methods</a></li><li><a class="toctext" href="constructors.html">Constructors</a></li><li><a class="toctext" href="conversion-and-promotion.html">Conversion and Promotion</a></li><li><a class="toctext" href="interfaces.html">Interfaces</a></li><li><a class="toctext" href="modules.html">Modules</a></li><li><a class="toctext" href="documentation.html">Documentation</a></li><li class="current"><a class="toctext" href="metaprogramming.html">Metaprogramming</a><ul class="internal"><li><a class="toctext" href="#Program-representation-1">Program representation</a></li><li><a class="toctext" href="#Expressions-and-evaluation-1">Expressions and evaluation</a></li><li><a class="toctext" href="#man-macros-1">Macros</a></li><li><a class="toctext" href="#Code-Generation-1">Code Generation</a></li><li><a class="toctext" href="#Non-Standard-String-Literals-1">Non-Standard String Literals</a></li><li><a class="toctext" href="#Generated-functions-1">Generated functions</a></li></ul></li><li><a class="toctext" href="arrays.html">Multi-dimensional Arrays</a></li><li><a class="toctext" href="linear-algebra.html">Linear algebra</a></li><li><a class="toctext" href="networking-and-streams.html">Networking and Streams</a></li><li><a class="toctext" href="parallel-computing.html">Parallel Computing</a></li><li><a class="toctext" href="dates.html">Date and DateTime</a></li><li><a class="toctext" href="interacting-with-julia.html">Interacting With Julia</a></li><li><a class="toctext" href="running-external-programs.html">Running External Programs</a></li><li><a class="toctext" href="calling-c-and-fortran-code.html">Calling C and Fortran Code</a></li><li><a class="toctext" href="handling-operating-system-variation.html">Handling Operating System Variation</a></li><li><a class="toctext" href="environment-variables.html">Environment Variables</a></li><li><a class="toctext" href="embedding.html">Embedding Julia</a></li><li><a class="toctext" href="packages.html">Packages</a></li><li><a class="toctext" href="profile.html">Profiling</a></li><li><a class="toctext" href="stacktraces.html">Stack Traces</a></li><li><a class="toctext" href="performance-tips.html">Performance Tips</a></li><li><a class="toctext" href="workflow-tips.html">Workflow Tips</a></li><li><a class="toctext" href="style-guide.html">Style Guide</a></li><li><a class="toctext" href="faq.html">Frequently Asked Questions</a></li><li><a class="toctext" href="noteworthy-differences.html">Noteworthy Differences from other Languages</a></li><li><a class="toctext" href="unicode-input.html">Unicode Input</a></li></ul></li><li><span class="toctext">Standard Library</span><ul><li><a class="toctext" href="../stdlib/base.html">Essentials</a></li><li><a class="toctext" href="../stdlib/collections.html">Collections and Data Structures</a></li><li><a class="toctext" href="../stdlib/math.html">Mathematics</a></li><li><a class="toctext" href="../stdlib/numbers.html">Numbers</a></li><li><a class="toctext" href="../stdlib/strings.html">Strings</a></li><li><a class="toctext" href="../stdlib/arrays.html">Arrays</a></li><li><a class="toctext" href="../stdlib/parallel.html">Tasks and Parallel Computing</a></li><li><a class="toctext" href="../stdlib/linalg.html">Linear Algebra</a></li><li><a class="toctext" href="../stdlib/constants.html">Constants</a></li><li><a class="toctext" href="../stdlib/file.html">Filesystem</a></li><li><a class="toctext" href="../stdlib/io-network.html">I/O and Network</a></li><li><a class="toctext" href="../stdlib/punctuation.html">Punctuation</a></li><li><a class="toctext" href="../stdlib/sort.html">Sorting and Related Functions</a></li><li><a class="toctext" href="../stdlib/pkg.html">Package Manager Functions</a></li><li><a class="toctext" href="../stdlib/dates.html">Dates and Time</a></li><li><a class="toctext" href="../stdlib/iterators.html">Iteration utilities</a></li><li><a class="toctext" href="../stdlib/test.html">Unit Testing</a></li><li><a class="toctext" href="../stdlib/c.html">C Interface</a></li><li><a class="toctext" href="../stdlib/libc.html">C Standard Library</a></li><li><a class="toctext" href="../stdlib/libdl.html">Dynamic Linker</a></li><li><a class="toctext" href="../stdlib/profile.html">Profiling</a></li><li><a class="toctext" href="../stdlib/stacktraces.html">StackTraces</a></li><li><a class="toctext" href="../stdlib/simd-types.html">SIMD Support</a></li></ul></li><li><span class="toctext">Developer Documentation</span><ul><li><a class="toctext" href="../devdocs/reflection.html">Reflection and introspection</a></li><li><span class="toctext">Documentation of Julia's Internals</span><ul><li><a class="toctext" href="../devdocs/init.html">Initialization of the Julia runtime</a></li><li><a class="toctext" href="../devdocs/ast.html">Julia ASTs</a></li><li><a class="toctext" href="../devdocs/types.html">More about types</a></li><li><a class="toctext" href="../devdocs/object.html">Memory layout of Julia Objects</a></li><li><a class="toctext" href="../devdocs/eval.html">Eval of Julia code</a></li><li><a class="toctext" href="../devdocs/callconv.html">Calling Conventions</a></li><li><a class="toctext" href="../devdocs/compiler.html">High-level Overview of the Native-Code Generation Process</a></li><li><a class="toctext" href="../devdocs/functions.html">Julia Functions</a></li><li><a class="toctext" href="../devdocs/cartesian.html">Base.Cartesian</a></li><li><a class="toctext" href="../devdocs/meta.html">Talking to the compiler (the <code>:meta</code> mechanism)</a></li><li><a class="toctext" href="../devdocs/subarrays.html">SubArrays</a></li><li><a class="toctext" href="../devdocs/sysimg.html">System Image Building</a></li><li><a class="toctext" href="../devdocs/llvm.html">Working with LLVM</a></li><li><a class="toctext" href="../devdocs/stdio.html">printf() and stdio in the Julia runtime</a></li><li><a class="toctext" href="../devdocs/boundscheck.html">Bounds checking</a></li><li><a class="toctext" href="../devdocs/locks.html">Proper maintenance and care of multi-threading locks</a></li><li><a class="toctext" href="../devdocs/offset-arrays.html">Arrays with custom indices</a></li><li><a class="toctext" href="../devdocs/libgit2.html">Base.LibGit2</a></li><li><a class="toctext" href="../devdocs/require.html">Module loading</a></li></ul></li><li><span class="toctext">Developing/debugging Julia's C code</span><ul><li><a class="toctext" href="../devdocs/backtraces.html">Reporting and analyzing crashes (segfaults)</a></li><li><a class="toctext" href="../devdocs/debuggingtips.html">gdb debugging tips</a></li><li><a class="toctext" href="../devdocs/valgrind.html">Using Valgrind with Julia</a></li><li><a class="toctext" href="../devdocs/sanitizers.html">Sanitizer support</a></li></ul></li></ul></li></ul></nav><article id="docs"><header><nav><ul><li>Manual</li><li><a href="metaprogramming.html">Metaprogramming</a></li></ul><a class="edit-page" href="https://github.com/JuliaLang/julia/tree/d386e40c17d43b79fc89d3e579fc04547241787c/doc/src/manual/metaprogramming.md"><span class="fa"></span> Edit on GitHub</a></nav><hr/><div id="topbar"><span>Metaprogramming</span><a class="fa fa-bars" href="#"></a></div></header><h1><a class="nav-anchor" id="Metaprogramming-1" href="#Metaprogramming-1">Metaprogramming</a></h1><p>The strongest legacy of Lisp in the Julia language is its metaprogramming support. Like Lisp, Julia represents its own code as a data structure of the language itself. Since code is represented by objects that can be created and manipulated from within the language, it is possible for a program to transform and generate its own code. This allows sophisticated code generation without extra build steps, and also allows true Lisp-style macros operating at the level of <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree">abstract syntax trees</a>. In contrast, preprocessor "macro" systems, like that of C and C++, perform textual manipulation and substitution before any actual parsing or interpretation occurs. Because all data types and code in Julia are represented by Julia data structures, powerful <a href="https://en.wikipedia.org/wiki/Reflection_%28computer_programming%29">reflection</a> capabilities are available to explore the internals of a program and its types just like any other data.</p><h2><a class="nav-anchor" id="Program-representation-1" href="#Program-representation-1">Program representation</a></h2><p>Every Julia program starts life as a string:</p><pre><code class="language-jldoctest">julia> prog = "1 + 1"
|
||
"1 + 1"</code></pre><p><strong>What happens next?</strong></p><p>The next step is to <a href="https://en.wikipedia.org/wiki/Parsing#Computer_languages">parse</a> each string into an object called an expression, represented by the Julia type <code>Expr</code>:</p><pre><code class="language-jldoctest">julia> ex1 = parse(prog)
|
||
:(1 + 1)
|
||
|
||
julia> typeof(ex1)
|
||
Expr</code></pre><p><code>Expr</code> objects contain two parts:</p><ul><li><p>a <code>Symbol</code> identifying the kind of expression. A symbol is an <a href="https://en.wikipedia.org/wiki/String_interning">interned string</a> identifier (more discussion below).</p></li></ul><pre><code class="language-jldoctest">julia> ex1.head
|
||
:call</code></pre><ul><li><p>the expression arguments, which may be symbols, other expressions, or literal values:</p></li></ul><pre><code class="language-jldoctest">julia> ex1.args
|
||
3-element Array{Any,1}:
|
||
:+
|
||
1
|
||
1</code></pre><p>Expressions may also be constructed directly in <a href="https://en.wikipedia.org/wiki/Polish_notation">prefix notation</a>:</p><pre><code class="language-jldoctest">julia> ex2 = Expr(:call, :+, 1, 1)
|
||
:(1 + 1)</code></pre><p>The two expressions constructed above – by parsing and by direct construction – are equivalent:</p><pre><code class="language-jldoctest">julia> ex1 == ex2
|
||
true</code></pre><p><strong>The key point here is that Julia code is internally represented as a data structure that is accessible from the language itself.</strong></p><p>The <a href="../stdlib/io-network.html#Base.dump"><code>dump()</code></a> function provides indented and annotated display of <code>Expr</code> objects:</p><pre><code class="language-jldoctest">julia> dump(ex2)
|
||
Expr
|
||
head: Symbol call
|
||
args: Array{Any}((3,))
|
||
1: Symbol +
|
||
2: Int64 1
|
||
3: Int64 1
|
||
typ: Any</code></pre><p><code>Expr</code> objects may also be nested:</p><pre><code class="language-jldoctest">julia> ex3 = parse("(4 + 4) / 2")
|
||
:((4 + 4) / 2)</code></pre><p>Another way to view expressions is with Meta.show_sexpr, which displays the <a href="https://en.wikipedia.org/wiki/S-expression">S-expression</a> form of a given <code>Expr</code>, which may look very familiar to users of Lisp. Here's an example illustrating the display on a nested <code>Expr</code>:</p><pre><code class="language-jldoctest">julia> Meta.show_sexpr(ex3)
|
||
(:call, :/, (:call, :+, 4, 4), 2)</code></pre><h3><a class="nav-anchor" id="Symbols-1" href="#Symbols-1">Symbols</a></h3><p>The <code>:</code> character has two syntactic purposes in Julia. The first form creates a <a href="../stdlib/strings.html#Core.Symbol"><code>Symbol</code></a>, an <a href="https://en.wikipedia.org/wiki/String_interning">interned string</a> used as one building-block of expressions:</p><pre><code class="language-julia-repl">julia> :foo
|
||
:foo
|
||
|
||
julia> typeof(ans)
|
||
Symbol</code></pre><p>The <a href="../stdlib/strings.html#Core.Symbol"><code>Symbol</code></a> constructor takes any number of arguments and creates a new symbol by concatenating their string representations together:</p><pre><code class="language-julia-repl">julia> :foo == Symbol("foo")
|
||
true
|
||
|
||
julia> Symbol("func",10)
|
||
:func10
|
||
|
||
julia> Symbol(:var,'_',"sym")
|
||
:var_sym</code></pre><p>In the context of an expression, symbols are used to indicate access to variables; when an expression is evaluated, a symbol is replaced with the value bound to that symbol in the appropriate <a href="variables-and-scoping.html#scope-of-variables-1">scope</a>.</p><p>Sometimes extra parentheses around the argument to <code>:</code> are needed to avoid ambiguity in parsing.:</p><pre><code class="language-julia-repl">julia> :(:)
|
||
:(:)
|
||
|
||
julia> :(::)
|
||
:(::)</code></pre><h2><a class="nav-anchor" id="Expressions-and-evaluation-1" href="#Expressions-and-evaluation-1">Expressions and evaluation</a></h2><h3><a class="nav-anchor" id="Quoting-1" href="#Quoting-1">Quoting</a></h3><p>The second syntactic purpose of the <code>:</code> character is to create expression objects without using the explicit <code>Expr</code> constructor. This is referred to as <em>quoting</em>. The <code>:</code> character, followed by paired parentheses around a single statement of Julia code, produces an <code>Expr</code> object based on the enclosed code. Here is example of the short form used to quote an arithmetic expression:</p><pre><code class="language-julia-repl">julia> ex = :(a+b*c+1)
|
||
:(a + b * c + 1)
|
||
|
||
julia> typeof(ex)
|
||
Expr</code></pre><p>(to view the structure of this expression, try <code>ex.head</code> and <code>ex.args</code>, or use <a href="../stdlib/io-network.html#Base.dump"><code>dump()</code></a> as above)</p><p>Note that equivalent expressions may be constructed using <a href="../stdlib/numbers.html#Base.parse-Tuple{Type,Any,Any}"><code>parse()</code></a> or the direct <code>Expr</code> form:</p><pre><code class="language-julia-repl">julia> :(a + b*c + 1) ==
|
||
parse("a + b*c + 1") ==
|
||
Expr(:call, :+, :a, Expr(:call, :*, :b, :c), 1)
|
||
true</code></pre><p>Expressions provided by the parser generally only have symbols, other expressions, and literal values as their args, whereas expressions constructed by Julia code can have arbitrary run-time values without literal forms as args. In this specific example, <code>+</code> and <code>a</code> are symbols, <code>*(b,c)</code> is a subexpression, and <code>1</code> is a literal 64-bit signed integer.</p><p>There is a second syntactic form of quoting for multiple expressions: blocks of code enclosed in <code>quote ... end</code>. Note that this form introduces <code>QuoteNode</code> elements to the expression tree, which must be considered when directly manipulating an expression tree generated from <code>quote</code> blocks. For other purposes, <code>:( ... )</code> and <code>quote .. end</code> blocks are treated identically.</p><pre><code class="language-julia-repl">julia> ex = quote
|
||
x = 1
|
||
y = 2
|
||
x + y
|
||
end
|
||
quote # none, line 2:
|
||
x = 1 # none, line 3:
|
||
y = 2 # none, line 4:
|
||
x + y
|
||
end
|
||
|
||
julia> typeof(ex)
|
||
Expr</code></pre><h3><a class="nav-anchor" id="Interpolation-1" href="#Interpolation-1">Interpolation</a></h3><p>Direct construction of <code>Expr</code> objects with value arguments is powerful, but <code>Expr</code> constructors can be tedious compared to "normal" Julia syntax. As an alternative, Julia allows "splicing" or interpolation of literals or expressions into quoted expressions. Interpolation is indicated by the <code>$</code> prefix.</p><p>In this example, the literal value of <code>a</code> is interpolated:</p><pre><code class="language-jldoctest">julia> a = 1;
|
||
|
||
julia> ex = :($a + b)
|
||
:(1 + b)</code></pre><p>Interpolating into an unquoted expression is not supported and will cause a compile-time error:</p><pre><code class="language-jlcodtest">julia> $a + b
|
||
ERROR: unsupported or misplaced expression $
|
||
...</code></pre><p>In this example, the tuple <code>(1,2,3)</code> is interpolated as an expression into a conditional test:</p><pre><code class="language-jldoctest">julia> ex = :(a in $:((1,2,3)) )
|
||
:(a in (1, 2, 3))</code></pre><p>Interpolating symbols into a nested expression requires enclosing each symbol in an enclosing quote block:</p><pre><code class="language-julia-repl">julia> :( :a in $( :(:a + :b) ) )
|
||
^^^^^^^^^^
|
||
quoted inner expression</code></pre><p>The use of <code>$</code> for expression interpolation is intentionally reminiscent of <a href="strings.html#string-interpolation-1">string interpolation</a> and <a href="running-external-programs.html#command-interpolation-1">command interpolation</a>. Expression interpolation allows convenient, readable programmatic construction of complex Julia expressions.</p><h3><a class="nav-anchor" id="[eval()](@ref)-and-effects-1" href="#[eval()](@ref)-and-effects-1"><a href="../stdlib/base.html#Core.eval"><code>eval()</code></a> and effects</a></h3><p>Given an expression object, one can cause Julia to evaluate (execute) it at global scope using <a href="../stdlib/base.html#Core.eval"><code>eval()</code></a>:</p><pre><code class="language-jldoctest">julia> :(1 + 2)
|
||
:(1 + 2)
|
||
|
||
julia> eval(ans)
|
||
3
|
||
|
||
julia> ex = :(a + b)
|
||
:(a + b)
|
||
|
||
julia> eval(ex)
|
||
ERROR: UndefVarError: b not defined
|
||
[...]
|
||
|
||
julia> a = 1; b = 2;
|
||
|
||
julia> eval(ex)
|
||
3</code></pre><p>Every <a href="modules.html#modules-1">module</a> has its own <a href="../stdlib/base.html#Core.eval"><code>eval()</code></a> function that evaluates expressions in its global scope. Expressions passed to <a href="../stdlib/base.html#Core.eval"><code>eval()</code></a> are not limited to returning values – they can also have side-effects that alter the state of the enclosing module's environment:</p><pre><code class="language-julia-repl">julia> ex = :(x = 1)
|
||
:(x = 1)
|
||
|
||
julia> x
|
||
ERROR: UndefVarError: x not defined
|
||
|
||
julia> eval(ex)
|
||
1
|
||
|
||
julia> x
|
||
1</code></pre><p>Here, the evaluation of an expression object causes a value to be assigned to the global variable <code>x</code>.</p><p>Since expressions are just <code>Expr</code> objects which can be constructed programmatically and then evaluated, it is possible to dynamically generate arbitrary code which can then be run using <a href="../stdlib/base.html#Core.eval"><code>eval()</code></a>. Here is a simple example:</p><pre><code class="language-julia-repl">julia> a = 1;
|
||
|
||
julia> ex = Expr(:call, :+, a, :b)
|
||
:(1 + b)
|
||
|
||
julia> a = 0; b = 2;
|
||
|
||
julia> eval(ex)
|
||
3</code></pre><p>The value of <code>a</code> is used to construct the expression <code>ex</code> which applies the <code>+</code> function to the value 1 and the variable <code>b</code>. Note the important distinction between the way <code>a</code> and <code>b</code> are used:</p><ul><li><p>The value of the <em>variable</em><code>a</code> at expression construction time is used as an immediate value in the expression. Thus, the value of <code>a</code> when the expression is evaluated no longer matters: the value in the expression is already <code>1</code>, independent of whatever the value of <code>a</code> might be.</p></li><li><p>On the other hand, the <em>symbol</em><code>:b</code> is used in the expression construction, so the value of the variable <code>b</code> at that time is irrelevant – <code>:b</code> is just a symbol and the variable <code>b</code> need not even be defined. At expression evaluation time, however, the value of the symbol <code>:b</code> is resolved by looking up the value of the variable <code>b</code>.</p></li></ul><h3><a class="nav-anchor" id="Functions-on-Expressions-1" href="#Functions-on-Expressions-1">Functions on <code>Expr</code>essions</a></h3><p>As hinted above, one extremely useful feature of Julia is the capability to generate and manipulate Julia code within Julia itself. We have already seen one example of a function returning <code>Expr</code> objects: the <a href="../stdlib/numbers.html#Base.parse-Tuple{Type,Any,Any}"><code>parse()</code></a> function, which takes a string of Julia code and returns the corresponding <code>Expr</code>. A function can also take one or more <code>Expr</code> objects as arguments, and return another <code>Expr</code>. Here is a simple, motivating example:</p><pre><code class="language-julia-repl">julia> function math_expr(op, op1, op2)
|
||
expr = Expr(:call, op, op1, op2)
|
||
return expr
|
||
end
|
||
math_expr (generic function with 1 method)
|
||
|
||
julia> ex = math_expr(:+, 1, Expr(:call, :*, 4, 5))
|
||
:(1 + 4 * 5)
|
||
|
||
julia> eval(ex)
|
||
21</code></pre><p>As another example, here is a function that doubles any numeric argument, but leaves expressions alone:</p><pre><code class="language-julia-repl">julia> function make_expr2(op, opr1, opr2)
|
||
opr1f, opr2f = map(x -> isa(x, Number) ? 2*x : x, (opr1, opr2))
|
||
retexpr = Expr(:call, op, opr1f, opr2f)
|
||
return retexpr
|
||
end
|
||
make_expr2 (generic function with 1 method)
|
||
|
||
julia> make_expr2(:+, 1, 2)
|
||
:(2 + 4)
|
||
|
||
julia> ex = make_expr2(:+, 1, Expr(:call, :*, 5, 8))
|
||
:(2 + 5 * 8)
|
||
|
||
julia> eval(ex)
|
||
42</code></pre><h2><a class="nav-anchor" id="man-macros-1" href="#man-macros-1">Macros</a></h2><p>Macros provide a method to include generated code in the final body of a program. A macro maps a tuple of arguments to a returned <em>expression</em>, and the resulting expression is compiled directly rather than requiring a runtime <a href="../stdlib/base.html#Core.eval"><code>eval()</code></a> call. Macro arguments may include expressions, literal values, and symbols.</p><h3><a class="nav-anchor" id="Basics-1" href="#Basics-1">Basics</a></h3><p>Here is an extraordinarily simple macro:</p><pre><code class="language-jldoctest">julia> macro sayhello()
|
||
return :( println("Hello, world!") )
|
||
end
|
||
@sayhello (macro with 1 method)</code></pre><p>Macros have a dedicated character in Julia's syntax: the <code>@</code> (at-sign), followed by the unique name declared in a <code>macro NAME ... end</code> block. In this example, the compiler will replace all instances of <code>@sayhello</code> with:</p><pre><code class="language-julia">:( println("Hello, world!") )</code></pre><p>When <code>@sayhello</code> is entered in the REPL, the expression executes immediately, thus we only see the evaluation result:</p><pre><code class="language-jldoctest">julia> @sayhello()
|
||
Hello, world!</code></pre><p>Now, consider a slightly more complex macro:</p><pre><code class="language-jldoctest">julia> macro sayhello(name)
|
||
return :( println("Hello, ", $name) )
|
||
end
|
||
@sayhello (macro with 1 method)</code></pre><p>This macro takes one argument: <code>name</code>. When <code>@sayhello</code> is encountered, the quoted expression is <em>expanded</em> to interpolate the value of the argument into the final expression:</p><pre><code class="language-jldoctest">julia> @sayhello("human")
|
||
Hello, human</code></pre><p>We can view the quoted return expression using the function <a href="../stdlib/base.html#Base.macroexpand"><code>macroexpand()</code></a> (<strong>important note:</strong> this is an extremely useful tool for debugging macros):</p><pre><code class="language-jldoctest">julia> ex = macroexpand( :(@sayhello("human")) )
|
||
:((println)("Hello, ", "human"))
|
||
|
||
julia> typeof(ex)
|
||
Expr</code></pre><p>We can see that the <code>"human"</code> literal has been interpolated into the expression.</p><p>There also exists a macro <a href="../stdlib/base.html#Base.@macroexpand"><code>@macroexpand</code></a> that is perhaps a bit more convenient than the <code>macroexpand</code> function:</p><pre><code class="language-jldoctest">julia> @macroexpand @sayhello "human"
|
||
:((println)("Hello, ", "human"))</code></pre><h3><a class="nav-anchor" id="Hold-up:-why-macros?-1" href="#Hold-up:-why-macros?-1">Hold up: why macros?</a></h3><p>We have already seen a function <code>f(::Expr...) -> Expr</code> in a previous section. In fact, <a href="../stdlib/base.html#Base.macroexpand"><code>macroexpand()</code></a> is also such a function. So, why do macros exist?</p><p>Macros are necessary because they execute when code is parsed, therefore, macros allow the programmer to generate and include fragments of customized code <em>before</em> the full program is run. To illustrate the difference, consider the following example:</p><pre><code class="language-jldoctest">julia> macro twostep(arg)
|
||
println("I execute at parse time. The argument is: ", arg)
|
||
return :(println("I execute at runtime. The argument is: ", $arg))
|
||
end
|
||
@twostep (macro with 1 method)
|
||
|
||
julia> ex = macroexpand( :(@twostep :(1, 2, 3)) );
|
||
I execute at parse time. The argument is: $(Expr(:quote, :((1, 2, 3))))</code></pre><p>The first call to <a href="../stdlib/io-network.html#Base.println"><code>println()</code></a> is executed when <a href="../stdlib/base.html#Base.macroexpand"><code>macroexpand()</code></a> is called. The resulting expression contains <em>only</em> the second <code>println</code>:</p><pre><code class="language-jldoctest">julia> typeof(ex)
|
||
Expr
|
||
|
||
julia> ex
|
||
:((println)("I execute at runtime. The argument is: ", $(Expr(:copyast, :($(QuoteNode(:((1, 2, 3)))))))))
|
||
|
||
julia> eval(ex)
|
||
I execute at runtime. The argument is: (1, 2, 3)</code></pre><h3><a class="nav-anchor" id="Macro-invocation-1" href="#Macro-invocation-1">Macro invocation</a></h3><p>Macros are invoked with the following general syntax:</p><pre><code class="language-julia">@name expr1 expr2 ...
|
||
@name(expr1, expr2, ...)</code></pre><p>Note the distinguishing <code>@</code> before the macro name and the lack of commas between the argument expressions in the first form, and the lack of whitespace after <code>@name</code> in the second form. The two styles should not be mixed. For example, the following syntax is different from the examples above; it passes the tuple <code>(expr1, expr2, ...)</code> as one argument to the macro:</p><pre><code class="language-julia">@name (expr1, expr2, ...)</code></pre><p>It is important to emphasize that macros receive their arguments as expressions, literals, or symbols. One way to explore macro arguments is to call the <a href="../stdlib/io-network.html#Base.show-Tuple{Any}"><code>show()</code></a> function within the macro body:</p><pre><code class="language-julia-repl">julia> macro showarg(x)
|
||
show(x)
|
||
# ... remainder of macro, returning an expression
|
||
end
|
||
@showarg (macro with 1 method)
|
||
|
||
julia> @showarg(a)
|
||
:a
|
||
|
||
julia> @showarg(1+1)
|
||
:(1 + 1)
|
||
|
||
julia> @showarg(println("Yo!"))
|
||
:(println("Yo!"))</code></pre><h3><a class="nav-anchor" id="Building-an-advanced-macro-1" href="#Building-an-advanced-macro-1">Building an advanced macro</a></h3><p>Here is a simplified definition of Julia's <code>@assert</code> macro:</p><pre><code class="language-jldoctest">julia> macro assert(ex)
|
||
return :( $ex ? nothing : throw(AssertionError($(string(ex)))) )
|
||
end
|
||
@assert (macro with 1 method)</code></pre><p>This macro can be used like this:</p><pre><code class="language-jldoctest">julia> @assert 1 == 1.0
|
||
|
||
julia> @assert 1 == 0
|
||
ERROR: AssertionError: 1 == 0</code></pre><p>In place of the written syntax, the macro call is expanded at parse time to its returned result. This is equivalent to writing:</p><pre><code class="language-julia">1 == 1.0 ? nothing : throw(AssertionError("1 == 1.0"))
|
||
1 == 0 ? nothing : throw(AssertionError("1 == 0"))</code></pre><p>That is, in the first call, the expression <code>:(1 == 1.0)</code> is spliced into the test condition slot, while the value of <code>string(:(1 == 1.0))</code> is spliced into the assertion message slot. The entire expression, thus constructed, is placed into the syntax tree where the <code>@assert</code> macro call occurs. Then at execution time, if the test expression evaluates to true, then <code>nothing</code> is returned, whereas if the test is false, an error is raised indicating the asserted expression that was false. Notice that it would not be possible to write this as a function, since only the <em>value</em> of the condition is available and it would be impossible to display the expression that computed it in the error message.</p><p>The actual definition of <code>@assert</code> in the standard library is more complicated. It allows the user to optionally specify their own error message, instead of just printing the failed expression. Just like in functions with a variable number of arguments, this is specified with an ellipses following the last argument:</p><pre><code class="language-jldoctest">julia> macro assert(ex, msgs...)
|
||
msg_body = isempty(msgs) ? ex : msgs[1]
|
||
msg = string(msg_body)
|
||
return :($ex ? nothing : throw(AssertionError($msg)))
|
||
end
|
||
@assert (macro with 1 method)</code></pre><p>Now <code>@assert</code> has two modes of operation, depending upon the number of arguments it receives! If there's only one argument, the tuple of expressions captured by <code>msgs</code> will be empty and it will behave the same as the simpler definition above. But now if the user specifies a second argument, it is printed in the message body instead of the failing expression. You can inspect the result of a macro expansion with the aptly named <a href="../stdlib/base.html#Base.macroexpand"><code>macroexpand()</code></a> function:</p><pre><code class="language-jldoctest">julia> macroexpand(:(@assert a == b))
|
||
:(if a == b
|
||
nothing
|
||
else
|
||
(throw)((AssertionError)("a == b"))
|
||
end)
|
||
|
||
julia> macroexpand(:(@assert a==b "a should equal b!"))
|
||
:(if a == b
|
||
nothing
|
||
else
|
||
(throw)((AssertionError)("a should equal b!"))
|
||
end)</code></pre><p>There is yet another case that the actual <code>@assert</code> macro handles: what if, in addition to printing "a should equal b," we wanted to print their values? One might naively try to use string interpolation in the custom message, e.g., <code>@assert a==b "a ($a) should equal b ($b)!"</code>, but this won't work as expected with the above macro. Can you see why? Recall from <a href="strings.html#string-interpolation-1">string interpolation</a> that an interpolated string is rewritten to a call to <a href="../stdlib/strings.html#Base.string"><code>string()</code></a>. Compare:</p><pre><code class="language-julia-repl">julia> typeof(:("a should equal b"))
|
||
String
|
||
|
||
julia> typeof(:("a ($a) should equal b ($b)!"))
|
||
Expr
|
||
|
||
julia> dump(:("a ($a) should equal b ($b)!"))
|
||
Expr
|
||
head: Symbol string
|
||
args: Array{Any}((5,))
|
||
1: String "a ("
|
||
2: Symbol a
|
||
3: String ") should equal b ("
|
||
4: Symbol b
|
||
5: String ")!"
|
||
typ: Any</code></pre><p>So now instead of getting a plain string in <code>msg_body</code>, the macro is receiving a full expression that will need to be evaluated in order to display as expected. This can be spliced directly into the returned expression as an argument to the <a href="../stdlib/strings.html#Base.string"><code>string()</code></a> call; see <a href="https://github.com/JuliaLang/julia/blob/master/base/error.jl"><code>error.jl</code></a> for the complete implementation.</p><p>The <code>@assert</code> macro makes great use of splicing into quoted expressions to simplify the manipulation of expressions inside the macro body.</p><h3><a class="nav-anchor" id="Hygiene-1" href="#Hygiene-1">Hygiene</a></h3><p>An issue that arises in more complex macros is that of <a href="https://en.wikipedia.org/wiki/Hygienic_macro">hygiene</a>. In short, macros must ensure that the variables they introduce in their returned expressions do not accidentally clash with existing variables in the surrounding code they expand into. Conversely, the expressions that are passed into a macro as arguments are often <em>expected</em> to evaluate in the context of the surrounding code, interacting with and modifying the existing variables. Another concern arises from the fact that a macro may be called in a different module from where it was defined. In this case we need to ensure that all global variables are resolved to the correct module. Julia already has a major advantage over languages with textual macro expansion (like C) in that it only needs to consider the returned expression. All the other variables (such as <code>msg</code> in <code>@assert</code> above) follow the <a href="variables-and-scoping.html#scope-of-variables-1">normal scoping block behavior</a>.</p><p>To demonstrate these issues, let us consider writing a <code>@time</code> macro that takes an expression as its argument, records the time, evaluates the expression, records the time again, prints the difference between the before and after times, and then has the value of the expression as its final value. The macro might look like this:</p><pre><code class="language-julia">macro time(ex)
|
||
return quote
|
||
local t0 = time()
|
||
local val = $ex
|
||
local t1 = time()
|
||
println("elapsed time: ", t1-t0, " seconds")
|
||
val
|
||
end
|
||
end</code></pre><p>Here, we want <code>t0</code>, <code>t1</code>, and <code>val</code> to be private temporary variables, and we want <code>time</code> to refer to the <a href="../stdlib/base.html#Base.Libc.time-Tuple{}"><code>time()</code></a> function in the standard library, not to any <code>time</code> variable the user might have (the same applies to <code>println</code>). Imagine the problems that could occur if the user expression <code>ex</code> also contained assignments to a variable called <code>t0</code>, or defined its own <code>time</code> variable. We might get errors, or mysteriously incorrect behavior.</p><p>Julia's macro expander solves these problems in the following way. First, variables within a macro result are classified as either local or global. A variable is considered local if it is assigned to (and not declared global), declared local, or used as a function argument name. Otherwise, it is considered global. Local variables are then renamed to be unique (using the <a href="../stdlib/base.html#Base.gensym"><code>gensym()</code></a> function, which generates new symbols), and global variables are resolved within the macro definition environment. Therefore both of the above concerns are handled; the macro's locals will not conflict with any user variables, and <code>time</code> and <code>println</code> will refer to the standard library definitions.</p><p>One problem remains however. Consider the following use of this macro:</p><pre><code class="language-julia">module MyModule
|
||
import Base.@time
|
||
|
||
time() = ... # compute something
|
||
|
||
@time time()
|
||
end</code></pre><p>Here the user expression <code>ex</code> is a call to <code>time</code>, but not the same <code>time</code> function that the macro uses. It clearly refers to <code>MyModule.time</code>. Therefore we must arrange for the code in <code>ex</code> to be resolved in the macro call environment. This is done by "escaping" the expression with <a href="../stdlib/base.html#Base.esc"><code>esc()</code></a>:</p><pre><code class="language-julia">macro time(ex)
|
||
...
|
||
local val = $(esc(ex))
|
||
...
|
||
end</code></pre><p>An expression wrapped in this manner is left alone by the macro expander and simply pasted into the output verbatim. Therefore it will be resolved in the macro call environment.</p><p>This escaping mechanism can be used to "violate" hygiene when necessary, in order to introduce or manipulate user variables. For example, the following macro sets <code>x</code> to zero in the call environment:</p><pre><code class="language-julia-repl">julia> macro zerox()
|
||
return esc(:(x = 0))
|
||
end
|
||
@zerox (macro with 1 method)
|
||
|
||
julia> function foo()
|
||
x = 1
|
||
@zerox
|
||
return x # is zero
|
||
end
|
||
foo (generic function with 1 method)
|
||
|
||
julia> foo()
|
||
0</code></pre><p>This kind of manipulation of variables should be used judiciously, but is occasionally quite handy.</p><h2><a class="nav-anchor" id="Code-Generation-1" href="#Code-Generation-1">Code Generation</a></h2><p>When a significant amount of repetitive boilerplate code is required, it is common to generate it programmatically to avoid redundancy. In most languages, this requires an extra build step, and a separate program to generate the repetitive code. In Julia, expression interpolation and <a href="../stdlib/base.html#Core.eval"><code>eval()</code></a> allow such code generation to take place in the normal course of program execution. For example, the following code defines a series of operators on three arguments in terms of their 2-argument forms:</p><pre><code class="language-julia">for op = (:+, :*, :&, :|, :$)
|
||
eval(quote
|
||
($op)(a,b,c) = ($op)(($op)(a,b),c)
|
||
end)
|
||
end</code></pre><p>In this manner, Julia acts as its own <a href="https://en.wikipedia.org/wiki/Preprocessor">preprocessor</a>, and allows code generation from inside the language. The above code could be written slightly more tersely using the <code>:</code> prefix quoting form:</p><pre><code class="language-julia">for op = (:+, :*, :&, :|, :$)
|
||
eval(:(($op)(a,b,c) = ($op)(($op)(a,b),c)))
|
||
end</code></pre><p>This sort of in-language code generation, however, using the <code>eval(quote(...))</code> pattern, is common enough that Julia comes with a macro to abbreviate this pattern:</p><pre><code class="language-julia">for op = (:+, :*, :&, :|, :$)
|
||
@eval ($op)(a,b,c) = ($op)(($op)(a,b),c)
|
||
end</code></pre><p>The <a href="../stdlib/base.html#Base.@eval"><code>@eval</code></a> macro rewrites this call to be precisely equivalent to the above longer versions. For longer blocks of generated code, the expression argument given to <a href="../stdlib/base.html#Base.@eval"><code>@eval</code></a> can be a block:</p><pre><code class="language-julia">@eval begin
|
||
# multiple lines
|
||
end</code></pre><h2><a class="nav-anchor" id="Non-Standard-String-Literals-1" href="#Non-Standard-String-Literals-1">Non-Standard String Literals</a></h2><p>Recall from <a href="strings.html#non-standard-string-literals-1">Strings</a> that string literals prefixed by an identifier are called non-standard string literals, and can have different semantics than un-prefixed string literals. For example:</p><ul><li><p><code>r"^\s*(?:#|$)"</code> produces a regular expression object rather than a string</p></li><li><p><code>b"DATA\xff\u2200"</code> is a byte array literal for <code>[68,65,84,65,255,226,136,128]</code>.</p></li></ul><p>Perhaps surprisingly, these behaviors are not hard-coded into the Julia parser or compiler. Instead, they are custom behaviors provided by a general mechanism that anyone can use: prefixed string literals are parsed as calls to specially-named macros. For example, the regular expression macro is just the following:</p><pre><code class="language-julia">macro r_str(p)
|
||
Regex(p)
|
||
end</code></pre><p>That's all. This macro says that the literal contents of the string literal <code>r"^\s*(?:#|$)"</code> should be passed to the <code>@r_str</code> macro and the result of that expansion should be placed in the syntax tree where the string literal occurs. In other words, the expression <code>r"^\s*(?:#|$)"</code> is equivalent to placing the following object directly into the syntax tree:</p><pre><code class="language-julia">Regex("^\\s*(?:#|\$)")</code></pre><p>Not only is the string literal form shorter and far more convenient, but it is also more efficient: since the regular expression is compiled and the <code>Regex</code> object is actually created <em>when the code is compiled</em>, the compilation occurs only once, rather than every time the code is executed. Consider if the regular expression occurs in a loop:</p><pre><code class="language-julia">for line = lines
|
||
m = match(r"^\s*(?:#|$)", line)
|
||
if m === nothing
|
||
# non-comment
|
||
else
|
||
# comment
|
||
end
|
||
end</code></pre><p>Since the regular expression <code>r"^\s*(?:#|$)"</code> is compiled and inserted into the syntax tree when this code is parsed, the expression is only compiled once instead of each time the loop is executed. In order to accomplish this without macros, one would have to write this loop like this:</p><pre><code class="language-julia">re = Regex("^\\s*(?:#|\$)")
|
||
for line = lines
|
||
m = match(re, line)
|
||
if m === nothing
|
||
# non-comment
|
||
else
|
||
# comment
|
||
end
|
||
end</code></pre><p>Moreover, if the compiler could not determine that the regex object was constant over all loops, certain optimizations might not be possible, making this version still less efficient than the more convenient literal form above. Of course, there are still situations where the non-literal form is more convenient: if one needs to interpolate a variable into the regular expression, one must take this more verbose approach; in cases where the regular expression pattern itself is dynamic, potentially changing upon each loop iteration, a new regular expression object must be constructed on each iteration. In the vast majority of use cases, however, regular expressions are not constructed based on run-time data. In this majority of cases, the ability to write regular expressions as compile-time values is invaluable.</p><p>Like non-standard string literals, non-standard command literals exist using a prefixed variant of the command literal syntax. The command literal <code>custom`literal`</code> is parsed as <code>@custom_cmd "literal"</code>. Julia itself does not contain any non-standard command literals, but packages can make use of this syntax. Aside from the different syntax and the <code>_cmd</code> suffix instead of the <code>_str</code> suffix, non-standard command literals behave exactly like non-standard string literals.</p><p>In the event that two modules provide non-standard string or command literals with the same name, it is possible to qualify the string or command literal with a module name. For instance, if both <code>Foo</code> and <code>Bar</code> provide non-standard string literal <code>@x_str</code>, then one can write <code>Foo.x"literal"</code> or <code>Bar.x"literal"</code> to disambiguate between the two.</p><p>The mechanism for user-defined string literals is deeply, profoundly powerful. Not only are Julia's non-standard literals implemented using it, but also the command literal syntax (<code>`echo "Hello, $person"`</code>) is implemented with the following innocuous-looking macro:</p><pre><code class="language-julia">macro cmd(str)
|
||
:(cmd_gen($(shell_parse(str)[1])))
|
||
end</code></pre><p>Of course, a large amount of complexity is hidden in the functions used in this macro definition, but they are just functions, written entirely in Julia. You can read their source and see precisely what they do – and all they do is construct expression objects to be inserted into your program's syntax tree.</p><h2><a class="nav-anchor" id="Generated-functions-1" href="#Generated-functions-1">Generated functions</a></h2><p>A very special macro is <code>@generated</code>, which allows you to define so-called <em>generated functions</em>. These have the capability to generate specialized code depending on the types of their arguments with more flexibility and/or less code than what can be achieved with multiple dispatch. While macros work with expressions at parsing-time and cannot access the types of their inputs, a generated function gets expanded at a time when the types of the arguments are known, but the function is not yet compiled.</p><p>Instead of performing some calculation or action, a generated function declaration returns a quoted expression which then forms the body for the method corresponding to the types of the arguments. When called, the body expression is first evaluated and compiled, then the returned expression is compiled and run. To make this efficient, the result is often cached. And to make this inferable, only a limited subset of the language is usable. Thus, generated functions provide a flexible framework to move work from run-time to compile-time, at the expense of greater restrictions on the allowable constructs.</p><p>When defining generated functions, there are four main differences to ordinary functions:</p><ol><li><p>You annotate the function declaration with the <code>@generated</code> macro. This adds some information to the AST that lets the compiler know that this is a generated function.</p></li><li><p>In the body of the generated function you only have access to the <em>types</em> of the arguments – not their values – and any function that was defined <em>before</em> the definition of the generated function.</p></li><li><p>Instead of calculating something or performing some action, you return a <em>quoted expression</em> which, when evaluated, does what you want.</p></li><li><p>Generated functions must not <em>mutate</em> or <em>observe</em> any non-constant global state (including, for example, IO, locks, non-local dictionaries, or using <code>method_exists</code>). This means they can only read global constants, and cannot have any side effects. In other words, they must be completely pure. Due to an implementation limitation, this also means that they currently cannot define a closure or untyped generator.</p></li></ol><p>It's easiest to illustrate this with an example. We can declare a generated function <code>foo</code> as</p><pre><code class="language-jldoctest">julia> @generated function foo(x)
|
||
Core.println(x)
|
||
return :(x * x)
|
||
end
|
||
foo (generic function with 1 method)</code></pre><p>Note that the body returns a quoted expression, namely <code>:(x * x)</code>, rather than just the value of <code>x * x</code>.</p><p>From the caller's perspective, they are very similar to regular functions; in fact, you don't have to know if you're calling a regular or generated function - the syntax and result of the call is just the same. Let's see how <code>foo</code> behaves:</p><pre><code class="language-jldoctest">julia> x = foo(2); # note: output is from println() statement in the body
|
||
Int64
|
||
|
||
julia> x # now we print x
|
||
4
|
||
|
||
julia> y = foo("bar");
|
||
String
|
||
|
||
julia> y
|
||
"barbar"</code></pre><p>So, we see that in the body of the generated function, <code>x</code> is the <em>type</em> of the passed argument, and the value returned by the generated function, is the result of evaluating the quoted expression we returned from the definition, now with the <em>value</em> of <code>x</code>.</p><p>What happens if we evaluate <code>foo</code> again with a type that we have already used?</p><pre><code class="language-jldoctest">julia> foo(4)
|
||
16</code></pre><p>Note that there is no printout of <a href="../stdlib/numbers.html#Core.Int64"><code>Int64</code></a>. We can see that the body of the generated function was only executed once here, for the specific set of argument types, and the result was cached. After that, for this example, the expression returned from the generated function on the first invocation was re-used as the method body. However, the actual caching behavior is an implementation-defined performance optimization, so it is invalid to depend too closely on this behavior.</p><p>The number of times a generated function is generated <em>might</em> be only once, but it <em>might</em> also be more often, or appear to not happen at all. As a consequence, you should <em>never</em> write a generated function with side effects - when, and how often, the side effects occur is undefined. (This is true for macros too - and just like for macros, the use of <a href="../stdlib/base.html#Core.eval"><code>eval()</code></a> in a generated function is a sign that you're doing something the wrong way.) However, unlike macros, the runtime system cannot correctly handle a call to <a href="../stdlib/base.html#Core.eval"><code>eval()</code></a>, so it is disallowed.</p><p>It is also important to see how <code>@generated</code> functions interact with method redefinition. Following the principle that a correct <code>@generated</code> function must not observe any mutable state or cause any mutation of global state, we see the following behavior. Observe that the generated function <em>cannot</em> call any method that was not defined prior to the <em>definition</em> of the generated function itself.</p><p>Initially <code>f(x)</code> has one definition</p><pre><code class="language-jldoctest">julia> f(x) = "original definition";</code></pre><p>Define other operations that use <code>f(x)</code>:</p><pre><code class="language-jldoctest">julia> g(x) = f(x);
|
||
|
||
julia> @generated gen1(x) = f(x);
|
||
|
||
julia> @generated gen2(x) = :(f(x));</code></pre><p>We now add some new definitions for <code>f(x)</code>:</p><pre><code class="language-jldoctest">julia> f(x::Int) = "definition for Int";
|
||
|
||
julia> f(x::Type{Int}) = "definition for Type{Int}";</code></pre><p>and compare how these results differ:</p><pre><code class="language-jldoctest">julia> f(1)
|
||
"definition for Int"
|
||
|
||
julia> g(1)
|
||
"definition for Int"
|
||
|
||
julia> gen1(1)
|
||
"original definition"
|
||
|
||
julia> gen2(1)
|
||
"definition for Int"</code></pre><p>Each method of a generated function has its own view of defined functions:</p><pre><code class="language-jldoctest">julia> @generated gen1(x::Real) = f(x);
|
||
|
||
julia> gen1(1)
|
||
"definition for Type{Int}"</code></pre><p>The example generated function <code>foo</code> above did not do anything a normal function <code>foo(x) = x * x</code> could not do (except printing the type on the first invocation, and incurring higher overhead). However, the power of a generated function lies in its ability to compute different quoted expressions depending on the types passed to it:</p><pre><code class="language-julia-repl">julia> @generated function bar(x)
|
||
if x <: Integer
|
||
return :(x ^ 2)
|
||
else
|
||
return :(x)
|
||
end
|
||
end
|
||
bar (generic function with 1 method)
|
||
|
||
julia> bar(4)
|
||
16
|
||
|
||
julia> bar("baz")
|
||
"baz"</code></pre><p>(although of course this contrived example would be more easily implemented using multiple dispatch...)</p><p>Abusing this will corrupt the runtime system and cause undefined behavior:</p><pre><code class="language-julia-repl">julia> @generated function baz(x)
|
||
if rand() < .9
|
||
return :(x^2)
|
||
else
|
||
return :("boo!")
|
||
end
|
||
end
|
||
baz (generic function with 1 method)</code></pre><p>Since the body of the generated function is non-deterministic, its behavior, <em>and the behavior of all subsequent code</em> is undefined.</p><p><em>Don't copy these examples!</em></p><p>These examples are hopefully helpful to illustrate how generated functions work, both in the definition end and at the call site; however, <em>don't copy them</em>, for the following reasons:</p><ul><li><p>the <code>foo</code> function has side-effects (the call to <code>Core.println</code>), and it is undefined exactly when, how often or how many times these side-effects will occur</p></li><li><p>the <code>bar</code> function solves a problem that is better solved with multiple dispatch - defining <code>bar(x) = x</code> and <code>bar(x::Integer) = x ^ 2</code> will do the same thing, but it is both simpler and faster.</p></li><li><p>the <code>baz</code> function is pathologically insane</p></li></ul><p>Note that the set of operations that should not be attempted in a generated function is unbounded, and the runtime system can currently only detect a subset of the invalid operations. There are many other operations that will simply corrupt the runtime system without notification, usually in subtle ways not obviously connected to the bad definition. Because the function generator is run during inference, it must respect all of the limitations of that code.</p><p>Some operations that should not be attempted include:</p><ol><li><p>Caching of native pointers.</p></li><li><p>Interacting with the contents or methods of Core.Inference in any way.</p></li><li><p>Observing any mutable state.</p><ul><li><p>Inference on the generated function may be run at <em>any</em> time, including while your code is attempting to observe or mutate this state.</p></li></ul></li><li><p>Taking any locks: C code you call out to may use locks internally, (for example, it is not problematic to call <code>malloc</code>, even though most implementations require locks internally) but don't attempt to hold or acquire any while executing Julia code.</p></li><li><p>Calling any function that is defined after the body of the generated function. This condition is relaxed for incrementally-loaded precompiled modules to allow calling any function in the module.</p></li></ol><p>Alright, now that we have a better understanding of how generated functions work, let's use them to build some more advanced (and valid) functionality...</p><h3><a class="nav-anchor" id="An-advanced-example-1" href="#An-advanced-example-1">An advanced example</a></h3><p>Julia's base library has a <a href="../stdlib/arrays.html#Base.sub2ind"><code>sub2ind()</code></a> function to calculate a linear index into an n-dimensional array, based on a set of n multilinear indices - in other words, to calculate the index <code>i</code> that can be used to index into an array <code>A</code> using <code>A[i]</code>, instead of <code>A[x,y,z,...]</code>. One possible implementation is the following:</p><pre><code class="language-jldoctest">julia> function sub2ind_loop(dims::NTuple{N}, I::Integer...) where N
|
||
ind = I[N] - 1
|
||
for i = N-1:-1:1
|
||
ind = I[i]-1 + dims[i]*ind
|
||
end
|
||
return ind + 1
|
||
end
|
||
sub2ind_loop (generic function with 1 method)
|
||
|
||
julia> sub2ind_loop((3, 5), 1, 2)
|
||
4</code></pre><p>The same thing can be done using recursion:</p><pre><code class="language-julia-repl">julia> sub2ind_rec(dims::Tuple{}) = 1;
|
||
|
||
julia> sub2ind_rec(dims::Tuple{}, i1::Integer, I::Integer...) =
|
||
i1 == 1 ? sub2ind_rec(dims, I...) : throw(BoundsError());
|
||
|
||
julia> sub2ind_rec(dims::Tuple{Integer, Vararg{Integer}}, i1::Integer) = i1;
|
||
|
||
julia> sub2ind_rec(dims::Tuple{Integer, Vararg{Integer}}, i1::Integer, I::Integer...) =
|
||
i1 + dims[1] * (sub2ind_rec(Base.tail(dims), I...) - 1);
|
||
|
||
julia> sub2ind_rec((3, 5), 1, 2)
|
||
4</code></pre><p>Both these implementations, although different, do essentially the same thing: a runtime loop over the dimensions of the array, collecting the offset in each dimension into the final index.</p><p>However, all the information we need for the loop is embedded in the type information of the arguments. Thus, we can utilize generated functions to move the iteration to compile-time; in compiler parlance, we use generated functions to manually unroll the loop. The body becomes almost identical, but instead of calculating the linear index, we build up an <em>expression</em> that calculates the index:</p><pre><code class="language-jldoctest">julia> @generated function sub2ind_gen(dims::NTuple{N}, I::Integer...) where N
|
||
ex = :(I[$N] - 1)
|
||
for i = (N - 1):-1:1
|
||
ex = :(I[$i] - 1 + dims[$i] * $ex)
|
||
end
|
||
return :($ex + 1)
|
||
end
|
||
sub2ind_gen (generic function with 1 method)
|
||
|
||
julia> sub2ind_gen((3, 5), 1, 2)
|
||
4</code></pre><p><strong>What code will this generate?</strong></p><p>An easy way to find out is to extract the body into another (regular) function:</p><pre><code class="language-jldoctest">julia> @generated function sub2ind_gen(dims::NTuple{N}, I::Integer...) where N
|
||
return sub2ind_gen_impl(dims, I...)
|
||
end
|
||
sub2ind_gen (generic function with 1 method)
|
||
|
||
julia> function sub2ind_gen_impl(dims::Type{T}, I...) where T <: NTuple{N,Any} where N
|
||
length(I) == N || return :(error("partial indexing is unsupported"))
|
||
ex = :(I[$N] - 1)
|
||
for i = (N - 1):-1:1
|
||
ex = :(I[$i] - 1 + dims[$i] * $ex)
|
||
end
|
||
return :($ex + 1)
|
||
end
|
||
sub2ind_gen_impl (generic function with 1 method)</code></pre><p>We can now execute <code>sub2ind_gen_impl</code> and examine the expression it returns:</p><pre><code class="language-jldoctest">julia> sub2ind_gen_impl(Tuple{Int,Int}, Int, Int)
|
||
:(((I[1] - 1) + dims[1] * (I[2] - 1)) + 1)</code></pre><p>So, the method body that will be used here doesn't include a loop at all - just indexing into the two tuples, multiplication and addition/subtraction. All the looping is performed compile-time, and we avoid looping during execution entirely. Thus, we only loop <em>once per type</em>, in this case once per <code>N</code> (except in edge cases where the function is generated more than once - see disclaimer above).</p><footer><hr/><a class="previous" href="documentation.html"><span class="direction">Previous</span><span class="title">Documentation</span></a><a class="next" href="arrays.html"><span class="direction">Next</span><span class="title">Multi-dimensional Arrays</span></a></footer></article></body></html>
|