(slide
 (title (center "Native thread support" ))
 (title (center "for Steel Bank Common Lisp"))
 (p (center " - or - "))
 (p (center
     "n things every programmer should know about signal handling"))
 (title " ")
 (p (center "Daniel Barlow"))
 (p (center "|metacircles|")))

#+nil
(slide
 (title "Personal ambition")
 (p "To achieve mastery of all I survey, by means of writing free
software in Common Lisp"))

#+nil
(slide
 (title "Personal ambition")
 (p "To achieve mastery of all I survey, by means of writing free
software in Common Lisp")
 (ul
  (li "Currently 11.7% accomplished")
  (li "Wrote quite a lot of software")
  (li "Narrowed scope considerably, by moving away from Primrose Hill")))

#+nil
(slide
 (title "Personal ambition")
 (p "To achieve mastery of all I survey, by means of writing free
software in Common Lisp")
 (ul
  (li "Currently 11.5% accomplished")
  (li "Wrote quite a lot of software")
  (li "Narrowed scope considerably, by moving away from Primrose Hill")))

(slide
 (title "Why Lisp?")
 (ul
  (li (line "Interactive" )
      (line "(Doesn't mean it's interpreted)"))
  (li (line "Extensible")
      (line "(Macros - which are not like C preprocessor macros - ")
      (line "allow definition of new language constructs)"))
  (li (line "Multiparadigm")
      (line "OO, functional, imperative, AOP, etc etc"))
  (li (line "Standard")
      (line "ANSI Common Lisp, (American National Standard X3.226)")
      (line "Which is surprisingly readable, for a language standard"))))


(slide
 (title "How?")
 (ul
  (li "Linux: commodity OS, commodity hardware")
  (li "... fast, reliable, cheap, blah")
  (li (line "Steel Bank Common Lisp: Lisp system based on CMU CL" )
      (line "20 years accumulated student project cruft, ")
      (line "which we're cleaning up"))
  (li (line "Not \"Lisp all the way down\" - but personally I'd")
      (line "rather not have to write my own ACPI code or IDE drivers anyway"))
  ))


;;; where they don't match


(slide
 (title "Calling Convention")
 (p "Lisp has exciting argument passing conventions")
 (pre "(defun foo (a b &rest rest &key (c 1) (d \"default\")) ...))")
 (pre "(defun bar (a b &optional (d \"default\")) ...)")
 (ul
  (li "we could shoehorn these into the official ABI, at some expense in speed")
  (li "or ignore the ABI for normal calls, and have special glue for calls to C")))


(slide 
 (title "Preprocessor")
 (p "Sometimes people do things that " (i "look") " like ordinary functions, but aren't")
 (ul
  (li (p (tt "errno"))
      (p (pre "#define errno (*__errno_location ())")))
  (li (p (tt "stat"))
      (smallpre ":; cat foo.c
#include <errno.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>

main() {
        struct stat st;
        return fstat(0,&st);
}
:; cc -o foo foo.c
:; nm --dynamic foo
08048490 R _IO_stdin_used
         w __deregister_frame_info
         U __fxstat
         w __gmon_start__
         U __libc_start_main
         w __register_frame_info"))))

(slide
 (title "Signal handling")
 (ul
  (li (line "Unix processes usually die if they get SIGSEGV, SIGBUS, or even SIGFPE.")
      (line "We don't"))
  (li "To debug interactively, we need")
  (ul (li "To find out where the error was (PC, register contents, etc)")
      (li "To restore the state properly after the debugger exits"))
  (li (line "And we're using some of these signals for non-fatal events anyway")
      (line "e.g. SIGSEGV for detecting writes to pages that haven't been GCed recently"))
  (li (line "SA_SIGINFO and three-argument signal handlers are reasonably ")
      (line "portable these days"))
  (li (tt "void (*sa_sigaction)(int, siginfo_t *, void *);"))))


;;; In this context, mention:
;;; floating point mode resetting
;;; sparc SA_SIGINFO lossage
;;; libraries that install signal handlers are bad news
;;; - sometimes they call the original handlers, sometimes they exit
;;; - if they do call the original handlers, they rarely get the arguments
;;;   right

;;; to add somewhere: introspection constraints (icache, SIGTRAP)

;;; --- Immediate challenge - threading


(slide
 (title "Once upon a time")
 (p "The CL standard doesn't cover threading interfaces, even though there
were at the time some Lisps which could run multiple tasks at once")
 (ul
  (li "These were the Lisp machines: single-user workstations in which
anything right down to the scheduler could be hooked")
  (li "Several Lisps since then have added userland threading (stack
groups, etc) and followed the Lispm interface, more or less.")
  (li (line "It has issues")
      (ul
       (li (line "Some operations intrinsically lead to scalability suckage")
	   (line "e.g. " (b "without-interrupts")))
       (li (line "Others can only be done efficiently when the scheduler is in Lisp")
	   (line "e.g. " (b "process-wait")))))
  (li (line "So, we ignore it")
      (line "Instead we have a very simple interface that looks a lot like the underlying system.")))
 (p "Some day we'll add a compatibility layer for porters.  No without-interrupts even so..."))
  
(slide
 (title "LinuxThreads: signals redux")
 (ul
  (li "There is some disconnect between the traditional Linux clone() model and
 the needs of POSIX threads")
  (li "LinuxThreads, to accomodate these, does strange things with signals to
ensure they get delivered to the threads that POSIX says they should")
  (li "We " (i "don't") " need our signal handling messed with")
  (li "And actually we don't even want this behaviour.  Lisp programmers
aren't going to care about the POSIX thread model")))

(slide
 (title "NPTL")
 (ul
  (li "NPTL is the New Posix Threading Library.  It takes a different 
approach to the Linux/POSIX disconnect, by adding flags to " (tt "clone()"))
  (li "But we were happy with the clone() model")
  (li "It also adds nice thread-local storage interfaces for C programmers")
  (li "But we're not C programmers (or only incidentally, anyway)")
  (li "It also involves some scalability work in the kernel to add support
for more threads, efficient locking (futexes), per-process GDT slots and
better scheduling")
  (li "This we " (i "can") " use")
  (li "Although currently we're targetting 2.4, so we don't.  But in a
future version...")))

(slide
 (title "Garbage collection, in 5 bullet points")
 (ul
  (li "stop all work")
  (li "starting with the machine registers, chase all the pointers we can see")
  (li "copy the objects we find into fresh storage")
  (li "free the old storage")
  (li "carry on"))
 (p "How do we do this?")
 (ul
  (li "Each thread has its own allocation region, so allocation within a 
region requires no locking until we fill a region") 
  (li "Use ptrace() to stop the threads and find their register contents")
  (li "The existing GC in SBCL can then be used mostly-unchanged")))

(slide
 (title "Special variables: the problem")
 (pre "(defvar *foo* 2) ; global variable *foo* ")
 (pre "(let ((*foo* 3))
  (format t \"*foo* is now ~A~%\" *foo*))")
 (p "meanwhile ...")
 (pre "(let ((*foo* 7))
  (format t \"*foo* is now ~A~%\" *foo*))")
 (li "Dynamic binding has to be per-thread if we are to remain sane" ))

(slide
 (title "Special variables: The solution")
 (ul
  (li "SBCL is shallow bound, and lots of existing SBCL code depends on this.
 We're not going to change this")
  (li "So, we add a slot to the symbol which contains a 'thread-local' value.
 This is an offset into some per-thread vector of storage")
  (li "We could use a register to hold the TLS base, but on X86 we don't
have many registers")
  (li "Or xMb-aligned stacks, and locate tls by masking the stack pointer
- but that eats a lot of address space")
  (li "So, we use the %fs segment selector")))

(slide
 (title "Locking")
 (ul
  (li "We have spinlocks (lock cmpxchg) for mostly internal use only")
  (li "User-intended locking primitives are based on queues")
  (li (line "If a thread can't get a lock, it puts itself to sleep on a queue")
      (line "(" (tt "sigtimedwait()") ") until woken by a signal.  Getting this right is")
      (line "subject to all kinds of interesting race conditions."))
  (li "Mutexes, recursive mutexes, condition variables.")
  (li "Timeouts")
  (li "Presently we're targetting 2.4, but futex and related stuff would
  be an obvious win here")))

;;; ---- here endeth the thread-specific section

(slide
 (title "Take-home messages: for kernel hackers")
 (ul
  (li "The whole world is not 'traditional Unix' command line tools.  Lisp
 may be unusual stuff, but more fashionable languages are starting to see
the same issues. e.g. java JIT, mono")
  (li "The clone() based task support in Linux is good for some much more
interesting stuff than POSIX threads.  Don't lose support for this in the
race to please the committees")
  (li "It would be nice if the third argument to signal handlers on SPARC
was a  ucontext like it is for every other arch we run on")))
  
(slide
 (title "Take-home messages: for library authors")
 (ul
  (li "We have tools that make life slightly easier: e.g. by compiling
small C programs to get the struct sizes and stuff out.  These work in
many cases")
  (li "Avoid funky cpp tricks : requires accessors in C, which need
hand-maintaining and are slow")
  (li (line "Write libraries, not frameworks")
      (line "Every time you want to add `convenience' features, ask if")
      (line "they're really as convenient for " (i "everybody") )
      (line "as you believe they'll be for you.  Expecial Bad news:")
      (ul
       (li "signal handling, signal handler installation")
       (li "requiring use of your own event loop")
       (li "calling exit()")))
  (li "Optional extra: machine-readable file describing functions and data structures.  
Maybe this is simply a matter of ensuring that SWIG can grok your header files")))

(slide
 (title "Where next?")
 (ul
  (li "Cleanup: TLS vector resizing, per-process stack sizes, UI stuff")
  (li "NPTL for threads: futex, furwocks, per-process GDT etc")
  (li (line "Ports to non-x86, non-linux")
      (line "non-x86 ports could use a real register for TLS, instead of a
segment register")
      (line "Some progress has reportedly been make on FreeBSD/x86, using rfork()"))
  (li  "64 bit support")
  (li "Applications")
  (li "Package repositories, code collections: CCLAN, cirCLe")))

(slide
 (title "Wrapping up")
 (p "For more information, downloads etc")
 (ul (li "http://sbcl.sourceforge.net/")
     (li "http://www.cliki.net/cclan")))