* C Stub Generator for Scheme -*- outline -*- The stubber is a set of macros that together generate a C stub file and expand to Scheme code for calling the C stubs. The macros constitute a language for expressing precisely how a C programmer would invoke a library, and what the library's invocation should look like in Scheme, without involving the low-level details of how any particular Scheme FFI works. This document is a scattered collection of notes on the design of the stubber. ** Implementation Notes The first order of business is to admit immediately that the implementation is a total mess, at least in s48-syntax.scm, by necessity of Scheme48's design. I do apologize. Furthermore, s48-codegen.scm ought to be thrown out along with the rest of Scheme48's basic FFI, but that's a separate issue. At least s48-codegen.scm will do a better job than one can do straightforwardly by hand. Consider, e.g., fixing the buggy { s48_value pair_s48 = (s48_cons ((s48_enter_integer (mumble)), (s48_enter_string (grumble)))); /* Do something with `pair_s48'. */ }, which may trigger a garbage collection in either `s48_enter_integer' or `s48_enter_string' and thereby invalidate the other argument on the stack. A first attempt to fix this might use a temporary variable: { s48_value pair_s48 = (s48_cons (S48_UNSPECIFIC, S48_UNSPECIFIC)); S48_DECLARE_GC_PROTECT (1); S48_GC_PROTECT_1 (pair_s48); S48_SET_CAR (pair_s48, (s48_enter_integer (mumble))); S48_SET_CDR (pair_s48, (s48_enter_string (grumble))); /* Do something with `pair_s48'. */ S48_GC_UNPROTECT (); }. It is left as an easy exercise for the reader to find what is still wrong even with the `fixed' code. It is left as a much trickier exercise for the reader to exorcise all code like this that litters Scheme48, scsh, and other hand-written stubs. *** Conversions, Expressions, Results, and Parameters There is currently some irksome confusion surrounding four (five, really) distinct concepts in the syntactic component: - conversion of data to and from C, - expressing Scheme values from C, - expressing Scheme *results* from C stub procedures, and - expressing C values from Scheme. Currently the words `expression' and `parameter' are misleadingly employed here for a variety of different purposes, and `result' for something else. [This paragraph warrants elaboration.] Fortunately, this doesn't bleed into the user-visible interface for the most part. Bizarrely, it even seems to work with the callback syntax, although s48-callback-syntax.scm is pretty confusing because of the way everything is named, which is mostly backwards for callbacks. Bug: (c-if foo (c-values a b) (c-values c d)) doesn't work, because conditionals don't match the continuation arity; instead it just makes unary continuations. This is very hairy to implement, though. *** System Calls In order to avoid masking mistakes, the macros SCHEME_VOID_SYSCALL and friends should be defined by default to cause errors. There should be internal macros that actually work. The DEFINE-UNIX-SYSCALL form should surround the body of the system call with #undef SCHEME_VOID_SYSCALL #define SCHEME_VOID_SYSCALL _SCHEME_VOID_SYSCALL ... #undef SCHEME_VOID_SYSCALL #define SCHEME_VOID_SYSCALL SCHEME_LOSING_VOID_SYSCALL *** Failure, Interruption, and Errors in System Calls When an error is detected within a C procedure called from Scheme, if the C procedure cannot handle the error, it should signal the error to Scheme. For example, if a system call fails for some reason other than interruption due to signal delivery, i.e. with an errno other than EINTR, it is often appropriate to signal a system error to Scheme. The naive way to do this is to longjmp out of the C procedure back into Scheme, or similar (e.g., exceptions in C++). This is what Scheme48 usually does, for example. What happens if the C procedure had stored in local variables resources for which Scheme had not taken responsibility, such as malloc'd storage, file descriptors, database handles, &c.? Before the longjmp, these would need to be released, or sequestered away somewhere, using some unwind protection mechanism. An alternative is to avoid non-local exits altogether, and to wrap every call to a C procedure with some overhead to check some error flag and optionally signal an error. Unfortunately, this gives no easy answer to the question of what the C procedure should do when the system call has failed. For example: (define-unix-syscall (%dup2-and-close (source-fd (c-integral "int")) (target-fd (c-integral "int")) (fd-pointer (c-alien-pointer "int"))) (c-begin "SCHEME_VOID_SYSCALL (dup2, (dup2 (source_fd, target_fd))); " "(*fd_pointer) = target_fd; " "SCHEME_VOID_SYSCALL (close, (close (source_fd))); " (c-unspecific))) This procedure, used to renumber a file descriptor, assumes that SCHEME_VOID_SYSCALL exits non-locally when the system call fails. If that were not the case, what would the procedure look like? For each system call, it always either restarts or returns normally (either with success or with an error). This way, there is only one path out of a system call. If you make a system call, and it fails, use SCHEME_SYSCALL_FAILURE to indicate the failure to Scheme: SCHEME_SYSCALL_FAILURE (syscall, errno, restart); where syscall is the name of the system call, errno is an expression for the error code, and restart is a command that restarts the system call. If SCHEME_SYSCALL_FAILURE returns, then you can report the error to Scheme with the C expression C-UNIX-SYSCALL-ERROR. Control can return back into Scheme, for instance to handle an interrupt, but must eventually return back either to restart the system call or to return from SCHEME_SYSCALL_FAILURE. SCHEME_VOID_SYSCALL, SCHEME_UINT_SYSCALL, and SCHEME_PTR_SYSCALL, all need to take an extra argument, for a command to be executed in the case of an error. **** Examples (define-unix-syscall (%dup2-and-close (source-fd (c-integral "int")) (target-fd (c-integral "int")) (fd-pointer (c-alien-pointer "int"))) (c-declare "int status = 0; " "int error = 0; ") (c-begin " SCHEME_VOID_SYSCALL " " (dup2, (dup2 (source_fd, target_fd)), " " do { status = 1; error = errno; goto finish; } while (0)); " " (*fd_pointer) = target_fd; " " SCHEME_VOID_SYSCALL " " (close, (close (source_fd)), " " do { status = 2; error = errno; goto finish; } while (0)); " " " "finish: " (c-cond ("status == 1" (c-unix-syscall-error "dup2" "errno")) ("status == 2" (c-unix-syscall-error "close" "errno")) (else (c-unspecific))))) Example of using SCHEME_SYSCALL_FAILURE directly, when more control is needed over the interpretation of error codes (some denote blocking conditions, not errors): (define-unix-syscall (%maybe-connect-socket (socket-fd (c-integral "int")) (address (c-immutable-byte-vector "address_bytes" "address_length"))) (c-declare "struct sockaddr *address = 0; " "int status = 0; " "int error = 0; ") (c-begin " address = ((struct sockaddr *) address_bytes); " "connect_loop: " " status = (connect (socket_fd, address, address_length)); " " if (status < 0) " " switch (errno) " " { " " case EAGAIN: " " case EINPROGRESS: " " case EWOULDBLOCK: " " status = 1; " " break; " " " " default: " " error = errno; " " SCHEME_SYSCALL_FAILURE (connect, error, goto connect_loop); " " } " (c-if "status < 0" (c-unix-syscall-error "connect" "error") (c-boolean "int" "status == 0")))) **** Problems There are some problems with this approach: 1. It is rapidly approaching a painful excess of bookkeeping. 2. It would be nice, after a system call failing with ENFILE or EMFILE, to run the garbage collector (once) before actually signalling the error. But running the garbage collector requires knowing where every pointer into the Scheme heap is. That means that any arguments passed as pointers to C (e.g., pointers into buffers represented in Scheme by byte vectors) will be invalid unless we are very clever and prohibit arithmetic on such pointers. This problem affects any approach to restart the system call without restarting the entire stub procedure, though. Is there a general way around this? 3. It is not clear how DEFINE-UNIX-{UINT,VOID}-SYSCALL should work now for the non-trivial cases. Should those cases be thrown out? If so, it is likely that DEFINE-UNIX-UINT-SYSCALL should be thrown out too, because it is not very useful. ** Copying Copyright (c) 2009, Taylor R. Campbell. Verbatim copying and distribution of this entire article are permitted worldwide, without royalty, in any medium, provided this notice, and the copyright notice, are preserved.