mollusk/jitty-scripts
1
1
Fork 0
Code Issues Pull Requests Releases Wiki Activity
jitty-scripts/julia-0.6.2/share/doc/julia/html/en/manual/calling-c-and-fortran-code.html

167 lines
71 KiB
HTML
Raw Normal View History

Add: julia-0.6.2 Former-commit-id: ccc667cf67d569f3fb3df39aa57c2134755a7551
2018-02-10 10:27:19 -07:00
<!DOCTYPE html>
<html lang="en"><head><meta charset="UTF-8"/><meta name="viewport" content="width=device-width, initial-scale=1.0"/><title>Calling C and Fortran Code · The Julia Language</title><script>(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-28835595-6', 'auto');
ga('send', 'pageview');
</script><link href="https://cdnjs.cloudflare.com/ajax/libs/normalize/4.2.0/normalize.min.css" rel="stylesheet" type="text/css"/><link href="https://fonts.googleapis.com/css?family=Lato|Roboto+Mono" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.6.3/css/font-awesome.min.css" rel="stylesheet" type="text/css"/><script>documenterBaseURL=".."</script><script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.2.0/require.min.js" data-main="../assets/documenter.js"></script><script src="../siteinfo.js"></script><script src="../../versions.js"></script><link href="../assets/highlightjs/default.css" rel="stylesheet" type="text/css"/><link href="../assets/documenter.css" rel="stylesheet" type="text/css"/></head><body><nav class="toc"><a href="../index.html"><img class="logo" src="../assets/logo.png" alt="The Julia Language logo"/></a><h1>The Julia Language</h1><select id="version-selector" onChange="window.location.href=this.value" style="visibility: hidden"></select><form class="search" action="../search.html"><input id="search-query" name="q" type="text" placeholder="Search docs"/></form><ul><li><a class="toctext" href="../index.html">Home</a></li><li><span class="toctext">Manual</span><ul><li><a class="toctext" href="introduction.html">Introduction</a></li><li><a class="toctext" href="getting-started.html">Getting Started</a></li><li><a class="toctext" href="variables.html">Variables</a></li><li><a class="toctext" href="integers-and-floating-point-numbers.html">Integers and Floating-Point Numbers</a></li><li><a class="toctext" href="mathematical-operations.html">Mathematical Operations and Elementary Functions</a></li><li><a class="toctext" href="complex-and-rational-numbers.html">Complex and Rational Numbers</a></li><li><a class="toctext" href="strings.html">Strings</a></li><li><a class="toctext" href="functions.html">Functions</a></li><li><a class="toctext" href="control-flow.html">Control Flow</a></li><li><a class="toctext" href="variables-and-scoping.html">Scope of Variables</a></li><li><a class="toctext" href="types.html">Types</a></li><li><a class="toctext" href="methods.html">Methods</a></li><li><a class="toctext" href="constructors.html">Constructors</a></li><li><a class="toctext" href="conversion-and-promotion.html">Conversion and Promotion</a></li><li><a class="toctext" href="interfaces.html">Interfaces</a></li><li><a class="toctext" href="modules.html">Modules</a></li><li><a class="toctext" href="documentation.html">Documentation</a></li><li><a class="toctext" href="metaprogramming.html">Metaprogramming</a></li><li><a class="toctext" href="arrays.html">Multi-dimensional Arrays</a></li><li><a class="toctext" href="linear-algebra.html">Linear algebra</a></li><li><a class="toctext" href="networking-and-streams.html">Networking and Streams</a></li><li><a class="toctext" href="parallel-computing.html">Parallel Computing</a></li><li><a class="toctext" href="dates.html">Date and DateTime</a></li><li><a class="toctext" href="interacting-with-julia.html">Interacting With Julia</a></li><li><a class="toctext" href="running-external-programs.html">Running External Programs</a></li><li class="current"><a class="toctext" href="calling-c-and-fortran-code.html">Calling C and Fortran Code</a><ul class="internal"><li><a class="toctext" href="#Creating-C-Compatible-Julia-Function-Pointers-1">Creating C-Compatible Julia Function Pointers</a></li><li><a class="toctext" href="#Mapping-C-Types-to-Julia-1">Mapping C Types to Julia</a></li><li><a class="toctext" href="#Mapping-C-Functions-to-Julia-1">Mapping C Functions to Julia</a></li><li><a class="toctext" href="#Some-Examples-of-C-Wrappers-1">Some Examples of C Wrappers</a></li><li><a class="toctext" href="#Garbage-Collection-Safety-1">Garbage Collection Safety</a></li><li><a class="toctext" href="#Non-constant-Function-Specifications-1">Non-constant Function Specifications</a></li><li><a class="toctext" href="#Indirect-Calls-1">Indirect Calls</a></li><li><a class="toctext" href="#Calling-Convention-1">Calling Convention</a></l
2292761
julia&gt; t
2292761
julia&gt; typeof(ans)
Int32</code></pre><p><code>clock</code> takes no arguments and returns an <a href="../stdlib/numbers.html#Core.Int32"><code>Int32</code></a>. One common gotcha is that a 1-tuple must be written with a trailing comma. For example, to call the <code>getenv</code> function to get a pointer to the value of an environment variable, one makes a call like this:</p><pre><code class="language-julia-repl">julia&gt; path = ccall((:getenv, &quot;libc&quot;), Cstring, (Cstring,), &quot;SHELL&quot;)
Cstring(@0x00007fff5fbffc45)
julia&gt; unsafe_string(path)
&quot;/bin/bash&quot;</code></pre><p>Note that the argument type tuple must be written as <code>(Cstring,)</code>, rather than <code>(Cstring)</code>. This is because <code>(Cstring)</code> is just the expression <code>Cstring</code> surrounded by parentheses, rather than a 1-tuple containing <code>Cstring</code>:</p><pre><code class="language-julia-repl">julia&gt; (Cstring)
Cstring
julia&gt; (Cstring,)
(Cstring,)</code></pre><p>In practice, especially when providing reusable functionality, one generally wraps <a href="../stdlib/c.html#ccall"><code>ccall</code></a> uses in Julia functions that set up arguments and then check for errors in whatever manner the C or Fortran function indicates them, propagating to the Julia caller as exceptions. This is especially important since C and Fortran APIs are notoriously inconsistent about how they indicate error conditions. For example, the <code>getenv</code> C library function is wrapped in the following Julia function, which is a simplified version of the actual definition from <a href="https://github.com/JuliaLang/julia/blob/master/base/env.jl"><code>env.jl</code></a>:</p><pre><code class="language-julia">function getenv(var::AbstractString)
val = ccall((:getenv, &quot;libc&quot;),
Cstring, (Cstring,), var)
if val == C_NULL
error(&quot;getenv: undefined variable: &quot;, var)
end
unsafe_string(val)
end</code></pre><p>The C <code>getenv</code> function indicates an error by returning <code>NULL</code>, but other standard C functions indicate errors in various different ways, including by returning -1, 0, 1 and other special values. This wrapper throws an exception clearly indicating the problem if the caller tries to get a non-existent environment variable:</p><pre><code class="language-julia-repl">julia&gt; getenv(&quot;SHELL&quot;)
&quot;/bin/bash&quot;
julia&gt; getenv(&quot;FOOBAR&quot;)
getenv: undefined variable: FOOBAR</code></pre><p>Here is a slightly more complex example that discovers the local machine&#39;s hostname:</p><pre><code class="language-julia">function gethostname()
hostname = Vector{UInt8}(128)
ccall((:gethostname, &quot;libc&quot;), Int32,
(Ptr{UInt8}, Csize_t),
hostname, sizeof(hostname))
hostname[end] = 0; # ensure null-termination
return unsafe_string(pointer(hostname))
end</code></pre><p>This example first allocates an array of bytes, then calls the C library function <code>gethostname</code> to fill the array in with the hostname, takes a pointer to the hostname buffer, and converts the pointer to a Julia string, assuming that it is a NUL-terminated C string. It is common for C libraries to use this pattern of requiring the caller to allocate memory to be passed to the callee and filled in. Allocation of memory from Julia like this is generally accomplished by creating an uninitialized array and passing a pointer to its data to the C function. This is why we don&#39;t use the <code>Cstring</code> type here: as the array is uninitialized, it could contain NUL bytes. Converting to a <code>Cstring</code> as part of the <a href="../stdlib/c.html#ccall"><code>ccall</code></a> checks for contained NUL bytes and could therefore throw a conversion error.</p><h2><a class="nav-anchor" id="Creating-C-Compatible-Julia-Function-Pointers-1" href="#Creating-C-Compatible-Julia-Function-Pointers-1">Creating C-Compatible Julia Function Pointers</a></h2><p>It is possible to pass Julia functions to native C functions that accept function pointer arguments. For example, to match C prototypes of the form:</p><pre><code class="language-c">typedef returntype (*functiontype)(argumenttype,...)</code></pre><p>The function <a href="../stdlib/c.html#Base.cfunction"><code>cfunction()</code></a> generates the C-compatible function pointer for a call to a Julia library function. Arguments to <a href="../stdlib/c.html#Base.cfunction"><code>cfunction()</code></a> are as follows:</p><ol><li><p>A Julia Function</p></li><li><p>Return type</p></li><li><p>A tuple of input types</p></li></ol><p>A classic example is the standard C library <code>qsort</code> function, declared as:</p><pre><code class="language-c">void qsort(void *base, size_t nmemb, size_t size,
int(*compare)(const void *a, const void *b));</code></pre><p>The <code>base</code> argument is a pointer to an array of length <code>nmemb</code>, with elements of <code>size</code> bytes each. <code>compare</code> is a callback function which takes pointers to two elements <code>a</code> and <code>b</code> and returns an integer less/greater than zero if <code>a</code> should appear before/after <code>b</code> (or zero if any order is permitted). Now, suppose that we have a 1d array <code>A</code> of values in Julia that we want to sort using the <code>qsort</code> function (rather than Julia&#39;s built-in <code>sort</code> function). Before we worry about calling <code>qsort</code> and passing arguments, we need to write a comparison function that works for some arbitrary type T:</p><pre><code class="language-jldoctest">julia&gt; function mycompare(a::T, b::T) where T
return convert(Cint, a &lt; b ? -1 : a &gt; b ? +1 : 0)::Cint
end
mycompare (generic function with 1 method)</code></pre><p>Notice that we have to be careful about the return type: <code>qsort</code> expects a function returning a C <code>int</code>, so we must be sure to return <code>Cint</code> via a call to <code>convert</code> and a <code>typeassert</code>.</p><p>In order to pass this function to C, we obtain its address using the function <code>cfunction</code>:</p><pre><code class="language-jldoctest">julia&gt; const mycompare_c = cfunction(mycompare, Cint, (Ref{Cdouble}, Ref{Cdouble}));</code></pre><p><a href="../stdlib/c.html#Base.cfunction"><code>cfunction()</code></a> accepts three arguments: the Julia function (<code>mycompare</code>), the return type (<code>Cint</code>), and a tuple of the argument types, in this case to sort an array of <code>Cdouble</code> (<a href="../stdlib/numbers.html#Core.Float64"><code>Float64</code></a>) elements.</p><p>The final call to <code>qsort</code> looks like this:</p><pre><code class="language-jldoctest">julia&gt; A = [1.3, -2.7, 4.4, 3.1]
4-element Array{Float64,1}:
1.3
-2.7
4.4
3.1
julia&gt; ccall(:qsort, Void, (Ptr{Cdouble}, Csize_t, Csize_t, Ptr{Void}),
A, length(A), sizeof(eltype(A)), mycompare_c)
julia&gt; A
4-element Array{Float64,1}:
-2.7
1.3
3.1
4.4</code></pre><p>As can be seen, <code>A</code> is changed to the sorted array <code>[-2.7, 1.3, 3.1, 4.4]</code>. Note that Julia knows how to convert an array into a <code>Ptr{Cdouble}</code>, how to compute the size of a type in bytes (identical to C&#39;s <code>sizeof</code> operator), and so on. For fun, try inserting a <code>println(&quot;mycompare($a,$b)&quot;)</code> line into <code>mycompare</code>, which will allow you to see the comparisons that <code>qsort</code> is performing (and to verify that it is really calling the Julia function that you passed to it).</p><h2><a class="nav-anchor" id="Mapping-C-Types-to-Julia-1" href="#Mapping-C-Types-to-Julia-1">Mapping C Types to Julia</a></h2><p>It is critical to exactly match the declared C type with its declaration in Julia. Inconsistencies can cause code that works correctly on one system to fail or produce indeterminate results on a different system.</p><p>Note that no C header files are used anywhere in the process of calling C functions: you are responsible for making sure that your Julia types and call signatures accurately reflect those in the C header file. (The <a href="https://github.com/ihnorton/Clang.jl">Clang package</a> can be used to auto-generate Julia code from a C header file.)</p><h3><a class="nav-anchor" id="Auto-conversion:-1" href="#Auto-conversion:-1">Auto-conversion:</a></h3><p>Julia automatically inserts calls to the <a href="../stdlib/c.html#Base.cconvert"><code>Base.cconvert()</code></a> function to convert each argument to the specified type. For example, the following call:</p><pre><code class="language-julia">ccall((:foo, &quot;libfoo&quot;), Void, (Int32, Float64), x, y)</code></pre><p>will behave as if the following were written:</p><pre><code class="language-julia">ccall((:foo, &quot;libfoo&quot;), Void, (Int32, Float64),
Base.unsafe_convert(Int32, Base.cconvert(Int32, x)),
Base.unsafe_convert(Float64, Base.cconvert(Float64, y)))</code></pre><p><a href="../stdlib/c.html#Base.cconvert"><code>Base.cconvert()</code></a> normally just calls <a href="../stdlib/base.html#Base.convert"><code>convert()</code></a>, but can be defined to return an arbitrary new object more appropriate for passing to C. For example, this is used to convert an <code>Array</code> of objects (e.g. strings) to an array of pointers.</p><p><a href="../stdlib/c.html#Base.unsafe_convert"><code>Base.unsafe_convert()</code></a> handles conversion to <code>Ptr</code> types. It is considered unsafe because converting an object to a native pointer can hide the object from the garbage collector, causing it to be freed prematurely.</p><h3><a class="nav-anchor" id="Type-Correspondences:-1" href="#Type-Correspondences:-1">Type Correspondences:</a></h3><p>First, a review of some relevant Julia type terminology:</p><table><tr><th>Syntax / Keyword</th><th>Example</th><th>Description</th></tr><tr><td><code>mutable struct</code></td><td><code>String</code></td><td>&quot;Leaf Type&quot; :: A group of related data that includes a type-tag, is managed by the Julia GC, and is defined by object-identity. The type parameters of a leaf type must be fully defined (no <code>TypeVars</code> are allowed) in order for the instance to be constructed.</td></tr><tr><td><code>abstract type</code></td><td><code>Any</code>, <code>AbstractArray{T, N}</code>, <code>Complex{T}</code></td><td>&quot;Super Type&quot; :: A super-type (not a leaf-type) that cannot be instantiated, but can be used to describe a group of types.</td></tr><tr><td><code>T{A}</code></td><td><code>Vector{Int}</code></td><td>&quot;Type Parameter&quot; :: A specialization of a type (typically used for dispatch or storage optimization).</td></tr><tr><td></td><td></td><td>&quot;TypeVar&quot; :: The <code>T</code> in the type parameter declaration is referred to as a TypeVar (short for type variable).</td></tr><tr><td><code>primitive type</code></td><td><code>Int</code>, <code>Float64</code></td><td>&quot;Primitive Type&quot; :: A type with no fields, but a size. It is stored and defined by-value.</td></tr><tr><td><code>struct</code></td><td><code>Pair{Int, Int}</code></td><td>&quot;Struct&quot; :: A type with all fields defined to be constant. It is defined by-value, and may be stored with a type-tag.</td></tr><tr><td></td><td><code>Complex128</code> (<code>isbits</code>)</td><td>&quot;Is-Bits&quot; :: A <code>primitive type</code>, or a <code>struct</code> type where all fields are other <code>isbits</code> types. It is defined by-value, and is stored without a type-tag.</td></tr><tr><td><code>struct ...; end</code></td><td><code>nothing</code></td><td>&quot;Singleton&quot; :: a Leaf Type or Struct with no fields.</td></tr><tr><td><code>(...)</code> or <code>tuple(...)</code></td><td><code>(1, 2, 3)</code></td><td>&quot;Tuple&quot; :: an immutable data-structure similar to an anonymous struct type, or a constant array. Represented as either an array or a struct.</td></tr></table><h3><a class="nav-anchor" id="Bits-Types:-1" href="#Bits-Types:-1">Bits Types:</a></h3><p>There are several special types to be aware of, as no other type can be defined to behave the same:</p><ul><li><p><code>Float32</code></p><p>Exactly corresponds to the <code>float</code> type in C (or <code>REAL*4</code> in Fortran).</p></li><li><p><code>Float64</code></p><p>Exactly corresponds to the <code>double</code> type in C (or <code>REAL*8</code> in Fortran).</p></li><li><p><code>Complex64</code></p><p>Exactly corresponds to the <code>complex float</code> type in C (or <code>COMPLEX*8</code> in Fortran).</p></li><li><p><code>Complex128</code></p><p>Exactly corresponds to the <code>complex double</code> type in C (or <code>COMPLEX*16</code> in Fortran).</p></li><li><p><code>Signed</code></p><p>Exactly corresponds to the <code>signed</code> type annotation in C (or any <code>INTEGER</code> type in Fortran). Any Julia type that is not a subtype of <a href="../stdlib/numbers.html#Core.Signed"><code>Signed</code></a>
ccall(:main, Int32, (Int32, Ptr{Ptr{UInt8}}), length(argv), argv)</code></pre></div></div><div class="admonition note"><div class="admonition-title">Note</div><div class="admonition-text"><p>A C function declared to return <code>Void</code> will return the value <code>nothing</code> in Julia.</p></div></div><h3><a class="nav-anchor" id="Struct-Type-correspondences-1" href="#Struct-Type-correspondences-1">Struct Type correspondences</a></h3><p>Composite types, aka <code>struct</code> in C or <code>TYPE</code> in Fortran90 (or <code>STRUCTURE</code> / <code>RECORD</code> in some variants of F77), can be mirrored in Julia by creating a <code>struct</code> definition with the same field layout.</p><p>When used recursively, <code>isbits</code> types are stored inline. All other types are stored as a pointer to the data. When mirroring a struct used by-value inside another struct in C, it is imperative that you do not attempt to manually copy the fields over, as this will not preserve the correct field alignment. Instead, declare an <code>isbits</code> struct type and use that instead. Unnamed structs are not possible in the translation to Julia.</p><p>Packed structs and union declarations are not supported by Julia.</p><p>You can get a near approximation of a <code>union</code> if you know, a priori, the field that will have the greatest size (potentially including padding). When translating your fields to Julia, declare the Julia field to be only of that type.</p><p>Arrays of parameters can be expressed with <code>NTuple</code>:</p><pre><code class="language-none">in C:
struct B {
int A[3];
};
b_a_2 = B.A[2];
in Julia:
struct B
A::NTuple{3, CInt}
end
b_a_2 = B.A[3] # note the difference in indexing (1-based in Julia, 0-based in C)</code></pre><p>Arrays of unknown size (C99-compliant variable length structs specified by <code>[]</code> or <code>[0]</code>) are not directly supported. Often the best way to deal with these is to deal with the byte offsets directly. For example, if a C library declared a proper string type and returned a pointer to it:</p><pre><code class="language-c">struct String {
int strlen;
char data[];
};</code></pre><p>In Julia, we can access the parts independently to make a copy of that string:</p><pre><code class="language-julia">str = from_c::Ptr{Void}
len = unsafe_load(Ptr{Cint}(str))
unsafe_string(str + Core.sizeof(Cint), len)</code></pre><h3><a class="nav-anchor" id="Type-Parameters-1" href="#Type-Parameters-1">Type Parameters</a></h3><p>The type arguments to <code>ccall</code> are evaluated statically, when the method containing the ccall is defined. They therefore must take the form of a literal tuple, not a variable, and cannot reference local variables.</p><p>This may sound like a strange restriction, but remember that since C is not a dynamic language like Julia, its functions can only accept argument types with a statically-known, fixed signature.</p><p>However, while the type layout must be known statically to compute the <code>ccall</code> ABI, the static parameters of the function are considered to be part of this static environment. The static parameters of the function may be used as type parameters in the <code>ccall</code> signature, as long as they don&#39;t affect the layout of the type. For example, <code>f(x::T) where {T} = ccall(:valid, Ptr{T}, (Ptr{T},), x)</code> is valid, since <code>Ptr</code> is always a word-size primitive type. But, <code>g(x::T) where {T} = ccall(:notvalid, T, (T,), x)</code> is not valid, since the type layout of <code>T</code> is not known statically.</p><h3><a class="nav-anchor" id="SIMD-Values-1" href="#SIMD-Values-1">SIMD Values</a></h3><p>Note: This feature is currently implemented on 64-bit x86 and AArch64 platforms only.</p><p>If a C/C++ routine has an argument or return value that is a native SIMD type, the corresponding Julia type is a homogeneous tuple of <code>VecElement</code> that naturally maps to the SIMD type. Specifically:</p><blockquote><ul><li><p>The tuple must be the same size as the SIMD type. For example, a tuple representing an <code>__m128</code> on x86 must have a size of 16 bytes.</p></li><li><p>The element type of the tuple must be an instance of <code>VecElement{T}</code> where <code>T</code> is a primitive type that is 1, 2, 4 or 8 bytes.</p></li></ul></blockquote><p>For instance, consider this C routine that uses AVX intrinsics:</p><pre><code class="language-c">#include &lt;immintrin.h&gt;
__m256 dist( __m256 a, __m256 b ) {
return _mm256_sqrt_ps(_mm256_add_ps(_mm256_mul_ps(a, a),
_mm256_mul_ps(b, b)));
}</code></pre><p>The following Julia code calls <code>dist</code> using <code>ccall</code>:</p><pre><code class="language-julia">const m256 = NTuple{8, VecElement{Float32}}
a = m256(ntuple(i -&gt; VecElement(sin(Float32(i))), 8))
b = m256(ntuple(i -&gt; VecElement(cos(Float32(i))), 8))
function call_dist(a::m256, b::m256)
ccall((:dist, &quot;libdist&quot;), m256, (m256, m256), a, b)
end
println(call_dist(a,b))</code></pre><p>The host machine must have the requisite SIMD registers. For example, the code above will not work on hosts without AVX support.</p><h3><a class="nav-anchor" id="Memory-Ownership-1" href="#Memory-Ownership-1">Memory Ownership</a></h3><p><strong>malloc/free</strong></p><p>Memory allocation and deallocation of such objects must be handled by calls to the appropriate cleanup routines in the libraries being used, just like in any C program. Do not try to free an object received from a C library with <code>Libc.free</code> in Julia, as this may result in the <code>free</code> function being called via the wrong <code>libc</code> library and cause Julia to crash. The reverse (passing an object allocated in Julia to be freed by an external library) is equally invalid.</p><h3><a class="nav-anchor" id="When-to-use-T,-Ptr{T}-and-Ref{T}-1" href="#When-to-use-T,-Ptr{T}-and-Ref{T}-1">When to use T, Ptr{T} and Ref{T}</a></h3><p>In Julia code wrapping calls to external C routines, ordinary (non-pointer) data should be declared to be of type <code>T</code> inside the <a href="../stdlib/c.html#ccall"><code>ccall</code></a>, as they are passed by value. For C code accepting pointers, <code>Ref{T}</code> should generally be used for the types of input arguments, allowing the use of pointers to memory managed by either Julia or C through the implicit call to <a href="../stdlib/c.html#Base.cconvert"><code>Base.cconvert()</code></a>. In contrast, pointers returned by the C function called should be declared to be of output type <code>Ptr{T}</code>, reflecting that the memory pointed to is managed by C only. Pointers contained in C structs should be represented as fields of type <code>Ptr{T}</code> within the corresponding Julia struct types designed to mimic the internal structure of corresponding C structs.</p><p>In Julia code wrapping calls to external Fortran routines, all input arguments should be declared as of type <code>Ref{T}</code>, as Fortran passes all variables by reference. The return type should either be <code>Void</code> for Fortran subroutines, or a <code>T</code> for Fortran functions returning the type <code>T</code>.</p><h2><a class="nav-anchor" id="Mapping-C-Functions-to-Julia-1" href="#Mapping-C-Functions-to-Julia-1">Mapping C Functions to Julia</a></h2><h3><a class="nav-anchor" id="ccall/cfunction-argument-translation-guide-1" href="#ccall/cfunction-argument-translation-guide-1"><code>ccall</code>/<code>cfunction</code> argument translation guide</a></h3><p>For translating a C argument list to Julia:</p><ul><li><p><code>T</code>, where <code>T</code> is one of the primitive types: <code>char</code>, <code>int</code>, <code>long</code>, <code>short</code>, <code>float</code>, <code>double</code>, <code>complex</code>, <code>enum</code> or any of their <code>typedef</code> equivalents</p><ul><li><p><code>T</code>, where <code>T</code> is an equivalent Julia Bits Type (per the table above)</p></li><li><p>if <code>T</code> is an <code>enum</code>, the argument type should be equivalent to <code>Cint</code> or <code>Cuint</code></p></li><li><p>argument value will be copied (passed by value)</p></li></ul></li><li><p><code>struct T</code> (including typedef to a struct)</p><ul><li><p><code>T</code>, where <code>T</code> is a Julia leaf type</p></li><li><p>argument value will be copied (passed by value)</p></li></ul></li><li><p><code>void*</code></p><ul><li><p>depends on how this parameter is used, first translate this to the intended pointer type, then determine the Julia equivalent using the remaining rules in this list</p></li><li><p>this argument may be declared as <code>Ptr{Void}</code>, if it really is just an unknown pointer</p></li></ul></li><li><p><code>jl_value_t*</code></p><ul><li><p><code>Any</code></p></li><li><p>argument value must be a valid Julia object</p></li><li><p>currently unsupported by <a href="../stdlib/c.html#Base.cfunction"><code>cfunction()</code></a></p></li></ul></li><li><p><code>jl_value_t**</code></p><ul><li><p><code>Ref{Any}</code></p></li><li><p>argumen
range = Ref{Cfloat}(0)
ccall(:foo, Void, (Ref{Cint}, Ref{Cfloat}), width, range)</code></pre><p>Upon return, the contents of <code>width</code> and <code>range</code> can be retrieved (if they were changed by <code>foo</code>) by <code>width[]</code> and <code>range[]</code>; that is, they act like zero-dimensional arrays.</p><h3><a class="nav-anchor" id="Special-Reference-Syntax-for-ccall-(deprecated):-1" href="#Special-Reference-Syntax-for-ccall-(deprecated):-1">Special Reference Syntax for ccall (deprecated):</a></h3><p>The <code>&amp;</code> syntax is deprecated, use the <code>Ref{T}</code> argument type instead.</p><p>A prefix <code>&amp;</code> is used on an argument to <a href="../stdlib/c.html#ccall"><code>ccall</code></a> to indicate that a pointer to a scalar argument should be passed instead of the scalar value itself (required for all Fortran function arguments, as noted above). The following example computes a dot product using a BLAS function.</p><pre><code class="language-julia">function compute_dot(DX::Vector{Float64}, DY::Vector{Float64})
@assert length(DX) == length(DY)
n = length(DX)
incx = incy = 1
product = ccall((:ddot_, &quot;libLAPACK&quot;),
Float64,
(Ptr{Int32}, Ptr{Float64}, Ptr{Int32}, Ptr{Float64}, Ptr{Int32}),
&amp;n, DX, &amp;incx, DY, &amp;incy)
return product
end</code></pre><p>The meaning of prefix <code>&amp;</code> is not quite the same as in C. In particular, any changes to the referenced variables will not be visible in Julia unless the type is mutable (declared via <code>type</code>). However, even for immutable structs it will not cause any harm for called functions to attempt such modifications (that is, writing through the passed pointers). Moreover, <code>&amp;</code> may be used with any expression, such as <code>&amp;0</code> or <code>&amp;f(x)</code>.</p><p>When a scalar value is passed with <code>&amp;</code> as an argument of type <code>Ptr{T}</code>, the value will first be converted to type <code>T</code>.</p><h2><a class="nav-anchor" id="Some-Examples-of-C-Wrappers-1" href="#Some-Examples-of-C-Wrappers-1">Some Examples of C Wrappers</a></h2><p>Here is a simple example of a C wrapper that returns a <code>Ptr</code> type:</p><pre><code class="language-julia">mutable struct gsl_permutation
end
# The corresponding C signature is
# gsl_permutation * gsl_permutation_alloc (size_t n);
function permutation_alloc(n::Integer)
output_ptr = ccall(
(:gsl_permutation_alloc, :libgsl), # name of C function and library
Ptr{gsl_permutation}, # output type
(Csize_t,), # tuple of input types
n # name of Julia variable to pass in
)
if output_ptr == C_NULL # Could not allocate memory
throw(OutOfMemoryError())
end
return output_ptr
end</code></pre><p>The <a href="https://www.gnu.org/software/gsl/">GNU Scientific Library</a> (here assumed to be accessible through <code>:libgsl</code>) defines an opaque pointer, <code>gsl_permutation *</code>, as the return type of the C function <code>gsl_permutation_alloc()</code>. As user code never has to look inside the <code>gsl_permutation</code> struct, the corresponding Julia wrapper simply needs a new type declaration, <code>gsl_permutation</code>, that has no internal fields and whose sole purpose is to be placed in the type parameter of a <code>Ptr</code> type. The return type of the <a href="../stdlib/c.html#ccall"><code>ccall</code></a> is declared as <code>Ptr{gsl_permutation}</code>, since the memory allocated and pointed to by <code>output_ptr</code> is controlled by C (and not Julia).</p><p>The input <code>n</code> is passed by value, and so the function&#39;s input signature is simply declared as <code>(Csize_t,)</code> without any <code>Ref</code> or <code>Ptr</code> necessary. (If the wrapper was calling a Fortran function instead, the corresponding function input signature should instead be <code>(Ref{Csize_t},)</code>, since Fortran variables are passed by reference.) Furthermore, <code>n</code> can be any type that is convertable to a <code>Csize_t</code> integer; the <a href="../stdlib/c.html#ccall"><code>ccall</code></a> implicitly calls <a href="../stdlib/c.html#Base.cconvert"><code>Base.cconvert(Csize_t, n)</code></a>.</p><p>Here is a second example wrapping the corresponding destructor:</p><pre><code class="language-julia"># The corresponding C signature is
# void gsl_permutation_free (gsl_permutation * p);
function permutation_free(p::Ref{gsl_permutation})
ccall(
(:gsl_permutation_free, :libgsl), # name of C function and library
Void, # output type
(Ref{gsl_permutation},), # tuple of input types
p # name of Julia variable to pass in
)
end</code></pre><p>Here, the input <code>p</code> is declared to be of type <code>Ref{gsl_permutation}</code>, meaning that the memory that <code>p</code> points to may be managed by Julia or by C. A pointer to memory allocated by C should be of type <code>Ptr{gsl_permutation}</code>, but it is convertable using <a href="../stdlib/c.html#Base.cconvert"><code>Base.cconvert()</code></a> and therefore can be used in the same (covariant) context of the input argument to a <a href="../stdlib/c.html#ccall"><code>ccall</code></a>. A pointer to memory allocated by Julia must be of type <code>Ref{gsl_permutation}</code>, to ensure that the memory address pointed to is valid and that Julia&#39;s garbage collector manages the chunk of memory pointed to correctly. Therefore, the <code>Ref{gsl_permutation}</code> declaration allows pointers managed by C or Julia to be used.</p><p>If the C wrapper never expects the user to pass pointers to memory managed by Julia, then using <code>p::Ptr{gsl_permutation}</code> for the method signature of the wrapper and similarly in the <a href="../stdlib/c.html#ccall"><code>ccall</code></a> is also acceptable.</p><p>Here is a third example passing Julia arrays:</p><pre><code class="language-julia"># The corresponding C signature is
# int gsl_sf_bessel_Jn_array (int nmin, int nmax, double x,
# double result_array[])
function sf_bessel_Jn_array(nmin::Integer, nmax::Integer, x::Real)
if nmax &lt; nmin
throw(DomainError())
end
result_array = Vector{Cdouble}(nmax - nmin + 1)
errorcode = ccall(
(:gsl_sf_bessel_Jn_array, :libgsl), # name of C function and library
Cint, # output type
(Cint, Cint, Cdouble, Ref{Cdouble}),# tuple of input types
nmin, nmax, x, result_array # names of Julia variables to pass in
)
if errorcode != 0
error(&quot;GSL error code $errorcode&quot;)
end
return result_array
end</code></pre><p>The C function wrapped returns an integer error code; the results of the actual evaluation of the Bessel J function populate the Julia array <code>result_array</code>. This variable can only be used with corresponding input type declaration <code>Ref{Cdouble}</code>, since its memory is allocated and managed by Julia, not C. The implicit call to <a href="../stdlib/c.html#Base.cconvert"><code>Base.cconvert(Ref{Cdouble}, result_array)</code></a> unpacks the Julia pointer to a Julia array data structure into a form understandable by C.</p><p>Note that for this code to work correctly, <code>result_array</code> must be declared to be of type <code>Ref{Cdouble}</code> and not <code>Ptr{Cdouble}</code>. The memory is managed by Julia and the <code>Ref</code> signature alerts Julia&#39;s garbage collector to keep managing the memory for <code>result_array</code> while the <a href="../stdlib/c.html#ccall"><code>ccall</code></a> executes. If <code>Ptr{Cdouble}</code> were used instead, the <a href="../stdlib/c.html#ccall"><code>ccall</code></a> may still work, but Julia&#39;s garbage collector would not be aware that the memory declared for <code>result_array</code> is being used by the external C function. As a result, the code may produce a memory leak if <code>result_array</code> never gets freed by the garbage collector, or if the garbage collector prematurely frees <code>result_array</code>, the C function may end up throwing an invalid memory access exception.</p><h2><a class="nav-anchor" id="Garbage-Collection-Safety-1" href="#Garbage-Collection-Safety-1">Garbage Collection Safety</a></h2><p>When passing data to a <a href="../stdlib/c.html#ccall"><code>ccall</code></a>, it is best to avoid using the <a href="../stdlib/c.html#Base.pointer"><code>pointer()</code></a> function. Instead define a convert method and pass the variables directly to the <a href="../stdlib/c.html#ccall"><code>ccall</code></a>. <a href="../stdlib/c.html#ccall"><code>ccall</code></a> automatically arranges that all of its arguments will be preserved from garbage collection until the call returns. If a C API will store a reference to memory allocated by Julia, after the <a href="../stdlib/c.html#ccall"><code>ccall</code></a> returns, you must arrange that the object remains visible to the garbage collector. The suggested way to handle this is to make a global variable of type <code>Array{Ref,1}</code> to hold these values, until the C library notifies you that it is finished with them.</p><p>Whenever you have created a pointer to Julia data, you must ensure the original data exists until you are done with using the pointer. Many methods in Julia such as <a href="../stdlib/c.html#Base.unsafe_load"><code>unsafe_load()</code></a> and <a href="../stdlib/strings.html#Core.String-Tuple{AbstractString}"><code>String()</code></a> make copies of data instead of taking ownership of the buffer, so that it is safe to free (or alter) the original data without affecting Julia. A notable exception is <a href="../stdlib/c.html#Base.unsafe_wrap-Union{Tuple{N}, Tuple{T}, Tuple{Union{Type{Array{T,N} where N}, Type{Array{T,N}}, Type{Array}},Ptr{T},Tuple{Vararg{Int64,N}}}} where N where T"><code>unsafe_wrap()</code></a> which, for performance reasons, shares (or can be told to take ownership of) the underlying buffer.</p><p>The garbage collector does not guarantee any order of finalization. That is, if <code>a</code> contained a reference to <code>b</code> and both <code>a</code> and <code>b</code> are due for garbage collection, there is no guarantee that <code>b</code> would be finalized after <code>a</code>. If proper finalization of <code>a</code> depends on <code>b</code> being valid, it must be handled in other ways.</p><h2><a class="nav-anchor" id="Non-constant-Function-Specifications-1" href="#Non-constant-Function-Specifications-1">Non-constant Function Specifications</a></h2><p>A <code>(name, library)</code> function specification must be a constant expression. However, it is possible to use computed values as function names by staging through
z, zlocal = gensym(string(func)), gensym()
eval(current_module(), :(global $z = C_NULL))
z = esc(z)
quote
let $zlocal::Ptr{Void} = $z::Ptr{Void}
if $zlocal == C_NULL
$zlocal = dlsym($(esc(lib))::Ptr{Void}, $(esc(func)))
global $z = $zlocal
end
$zlocal
end
end
end
mylibvar = Libdl.dlopen(&quot;mylib&quot;)
ccall(@dlsym(&quot;myfunc&quot;, mylibvar), Void, ())</code></pre><h2><a class="nav-anchor" id="Calling-Convention-1" href="#Calling-Convention-1">Calling Convention</a></h2><p>The second argument to <a href="../stdlib/c.html#ccall"><code>ccall</code></a> can optionally be a calling convention specifier (immediately preceding return type). Without any specifier, the platform-default C calling convention is used. Other supported conventions are: <code>stdcall</code>, <code>cdecl</code>, <code>fastcall</code>, and <code>thiscall</code>. For example (from <code>base/libc.jl</code>) we see the same <code>gethostname</code><a href="../stdlib/c.html#ccall"><code>ccall</code></a> as above, but with the correct signature for Windows:</p><pre><code class="language-julia">hn = Vector{UInt8}(256)
err = ccall(:gethostname, stdcall, Int32, (Ptr{UInt8}, UInt32), hn, length(hn))</code></pre><p>For more information, please see the <a href="http://llvm.org/docs/LangRef.html#calling-conventions">LLVM Language Reference</a>.</p><p>There is one additional special calling convention <code>llvmcall</code>, which allows inserting calls to LLVM intrinsics directly. This can be especially useful when targeting unusual platforms such as GPGPUs. For example, for <a href="http://llvm.org/docs/NVPTXUsage.html">CUDA</a>, we need to be able to read the thread index:</p><pre><code class="language-julia">ccall(&quot;llvm.nvvm.read.ptx.sreg.tid.x&quot;, llvmcall, Int32, ())</code></pre><p>As with any <code>ccall</code>, it is essential to get the argument signature exactly correct. Also, note that there is no compatibility layer that ensures the intrinsic makes sense and works on the current target, unlike the equivalent Julia functions exposed by <code>Core.Intrinsics</code>.</p><h2><a class="nav-anchor" id="Accessing-Global-Variables-1" href="#Accessing-Global-Variables-1">Accessing Global Variables</a></h2><p>Global variables exported by native libraries can be accessed by name using the <a href="../stdlib/c.html#Core.Intrinsics.cglobal"><code>cglobal()</code></a> function. The arguments to <a href="../stdlib/c.html#Core.Intrinsics.cglobal"><code>cglobal()</code></a> are a symbol specification identical to that used by <a href="../stdlib/c.html#ccall"><code>ccall</code></a>, and a type describing the value stored in the variable:</p><pre><code class="language-julia-repl">julia&gt; cglobal((:errno, :libc), Int32)
Ptr{Int32} @0x00007f418d0816b8</code></pre><p>The result is a pointer giving the address of the value. The value can be manipulated through this pointer using <a href="../stdlib/c.html#Base.unsafe_load"><code>unsafe_load()</code></a> and <a href="../stdlib/c.html#Base.unsafe_store!"><code>unsafe_store!()</code></a>.</p><h2><a class="nav-anchor" id="Accessing-Data-through-a-Pointer-1" href="#Accessing-Data-through-a-Pointer-1">Accessing Data through a Pointer</a></h2><p>The following methods are described as &quot;unsafe&quot; because a bad pointer or type declaration can cause Julia to terminate abruptly.</p><p>Given a <code>Ptr{T}</code>, the contents of type <code>T</code> can generally be copied from the referenced memory into a Julia object using <code>unsafe_load(ptr, [index])</code>. The index argument is optional (default is 1), and follows the Julia-convention of 1-based indexing. This function is intentionally similar to the behavior of <a href="../stdlib/arrays.html#Base.getindex-Tuple{Type,Vararg{Any,N} where N}"><code>getindex()</code></a> and <a href="../stdlib/arrays.html#Base.setindex!-Tuple{AbstractArray,Any,Vararg{Any,N} where N}"><code>setindex!()</code></a> (e.g. <code>[]</code> access syntax).</p><p>The return value will be a new object initialized to contain a copy of the contents of the referenced memory. The referenced memory can safely be freed or released.</p><p>If <code>T</code> is <code>Any</code>, then the memory is assumed to contain a reference to a Julia object (a <code>jl_value_t*</code>), the result will be a reference to this object, and the object will not be copied. You must be careful in this case to ensure that the object was always visible to the garbage collector (pointers do not count, but the new reference does) to ensure the memory is not prematurely freed. Note that if the object was not originally allocated by Julia, the new object will never be finalized by Julia&#39;s garbage collector. If the <code>Ptr</code> itself is actually a <code>jl_value_t*</code>, it can be converted back to a Julia object reference by <a href="../stdlib/c.html#Base.unsafe_pointer_to_objref"><code>unsafe_pointer_to_objref(ptr)</code></a>. (Julia values <code>v</code> can be converted to <code>jl_value_t*</code> pointers, as <code>Ptr{Void}</code>, by calling <a href="../stdlib/c.html#Base.pointer_from_objref"><code>pointer_from_objref(v)</code></a>.)</p><p>The reverse operation (writing data to a <code>Ptr{T}</code>), can be performed using <a href="../stdlib/c.html#Base.unsafe_store!"><code>unsafe_store!(ptr, value, [index])</code></a>. Currently, this is only supported for primitive types or other pointer-free (<code>isbits</code>) immutable struct types.</p><p>Any operation that throws an error is probably currently unimplemented and should be posted as a bug so that it can be resolved.</p><p>If the pointer of interest is a plain-data array (primitive type or immutable struct), the function <a href="../stdlib/c.html#Base.unsafe_wrap-Union{Tuple{N}, Tuple{T}, Tuple{Union{Type{Array{T,N} where N}, Type{Array{T,N}}, Type{Array}},Ptr{T},Tuple{Vararg{Int64,N}}}} where N where T"><code>unsafe_wrap(Array, ptr,dims,[own])</code></a> may be more useful. The final parameter should be true if Julia should &quot;take ownership&quot; of the underlying buffer and call <code>free(ptr)</code> when the returned <code>Array</code> object is finalized. If the <code>own</code> parameter is omitted or false, the caller must ensure the buffer remains in existence until all access is complete.</p><p>Arithmetic on the <code>Ptr</code> type in Julia (e.g. using <code>+</code>) does not behave the same as C&#39;s pointer arithmetic. Adding an integer to a <code>Ptr</code> in Julia always moves the pointer by some number of <em>bytes</em>, not elements. This way, the address values obtained from pointer arithmetic do not depend on the element types of pointers.</p><h2><a class="nav-anchor" id="Thread-safety-1" href="#Thread-safety-1">Thread-safety</a></h2><p>Some C libraries execute their callbacks from a different thre
wait(cond)</code></pre><p>The callback you pass to C should only execute a <a href="../stdlib/c.html#ccall"><code>ccall</code></a> to <code>:uv_async_send</code>, passing <code>cond.handle</code> as the argument, taking care to avoid any allocations or other interactions with the Julia runtime.</p><p>Note that events may be coalesced, so multiple calls to <code>uv_async_send</code> may result in a single wakeup notification to the condition.</p><h2><a class="nav-anchor" id="More-About-Callbacks-1" href="#More-About-Callbacks-1">More About Callbacks</a></h2><p>For more details on how to pass callbacks to C libraries, see this <a href="https://julialang.org/blog/2013/05/callback">blog post</a>.</p><h2><a class="nav-anchor" id="C-1" href="#C-1">C++</a></h2><p>For direct C++ interfacing, see the <a href="https://github.com/Keno/Cxx.jl">Cxx</a> package. For tools to create C++ bindings, see the <a href="https://github.com/JuliaInterop/CxxWrap.jl">CxxWrap</a> package.</p><footer><hr/><a class="previous" href="running-external-programs.html"><span class="direction">Previous</span><span class="title">Running External Programs</span></a><a class="next" href="handling-operating-system-variation.html"><span class="direction">Next</span><span class="title">Handling Operating System Variation</span></a></footer></article></body></html>
Reference in New Issue Copy Permalink