\documentclass[10pt]{beamer}
\include{macros}
\usepackage[active]{srcltx}
\mode
{
\usetheme{Warsaw}
%\usecolortheme{rose}
\usecolortheme{seahorse}
%\setbeamercovered{transparent}
% or whatever (possibly just delete it)
}
\usepackage[english]{babel}
% or whatever
\usepackage[latin1]{inputenc}
% or whatever
\usepackage{times}
\usepackage[T1]{fontenc}
% Or whatever. Note that the encoding and the font should match. If T1
% does not look nice, try deleting the line with the fontenc.
\title{SAGE Days 3: A Social and Technical Status Report}
\author{William Stein}
% - Use the \inst command only if there are several affiliations.
% - Keep it simple, no one is interested in your street address.
\date[Feb 17, 2007] % (optional)
{\vspace{-6ex}
February 17, 2007, SAGE Days 3
}
\subject{Talks}
% Delete this, if you do not want the table of contents to pop up at
% the beginning of each subsection:
\AtBeginSubsection[]
{
\begin{frame}
\frametitle{Outline}
\tableofcontents[currentsection,currentsubsection]
\end{frame}
}
% If you wish to uncover everything in a step-wise fashion, uncomment
% the following command:
%\beamerdefaultoverlayspecification{<+->}
\begin{document}
\begin{frame}
\titlepage
\end{frame}
\section{SAGE: Social Status Report}
\begin{frame}
\vfill
\begin{center}
{\dred \LARGE SAGE: Social Status Report}
\end{center}
\vfill
\end{frame}
\begin{frame}
\frametitle{The SAGE Goal: Mission Statement}
{\Large {\dred THE SAGE GOAL:} Create a viable completely free
open source alternative to Magma, Matlab, Maple, and Mathematica.}
\vfill
Thus SAGE aims to be the analogue for mathematics research of {\dred Firefox}
for web browsing and {\dred Linux} for operating systems.
{\em Science and mathematics is supposed to be done openly}.
\vfill
{\dred THE SAGE GOAL is incredibly frickin' hard} for a number of reasons. In
my opinion nobody has come close yet. When I think
about how hard this is, I realize it is basically
impossible.\\
{\dgreen Solution: Do not think about how hard it is.}
\vfill
But we {\dred must} create SAGE. Otherwise a few people and
corporations will {\em continue to exert undue control}
over mathematical research, especially as use
of computers in math research becomes increasingly
important (as it will).
\end{frame}
\begin{frame}
\frametitle{How big is SAGE?}
Short answer -- 85MB in source form.
\begin{itemize}
\item {\dred 57 standard packages:} This is close
to the limit of what I can maintain. I plan
to add a sparse
numerical linear algebra package (e.g., SuperLU),
pytables, lie, and scipy. People from the numerical
community like the SAGE package system, and we are
combining forces.
\vfill
\item {\dred 27 optional packages:}
Lesson learned --
\begin{enumerate}
\item The build-from-source optional packages are a
{\bf maintenance nightmare}, since testing that they
work on a wide range of systems is incredibly hard.
\item
In the long run all build-from-source
optional package should either
disappear or become standard.
(The ones that don't should be
installed via standard tools, e.g., fink, apt-get,
Cygwin, etc. )
\item
The goal with SAGE is having a {\em complete
uniform well-tested working system}, and optional
packages break this, since often they do {\bf not} just work.
\end{enumerate}
\item The {\dred core SAGE library} contains
{\dred 115,322 unique} lines of code and docstrings.
This is still quite manageable.
\end{itemize}
\end{frame}
\begin{frame}
\frametitle{The Current Active Developers}
\begin{itemize}
\item {\dred 21 people have had patches
to the SAGE library accepted during the last
two months (many have submitted thousands
of lines of code):}\\
\vfill
Martin Albrecht,
Nick Alexander,
Tom Boothby,
Robert Bradshaw,
Iftikhar A. Burhanuddin,
Alex Clemesha,
Didier Deshomme,
David Harvey,
Josh Kantor,
Emily Kirkman,
David Kohel,
Robert Miller,
Joel B. Mohler,
Bobby Moretti,
Andrey Novoseltsev,
Dorian Raymer,
Yi Qiang,
Steven Sivek,
Jaap Spies,
William Stein,
Carl Witty...\\
\vfill
This is a huge amount
of work. Applause!
\vfill
\item Many people have contributed
packages, bug fixes to packages, code snippets, etc.
\end{itemize}
\end{frame}
\begin{frame}
\frametitle{Growing Pains}
\begin{itemize}
\item I spend a {\dred huge amount of time} just
getting patches to {\dred work correctly}
and have enough doctests, etc. Soon I will
be very busy with teaching, etc., and won't
always be able to do this at the currently level.
\vfill
\item So establishing a {\dred patch
refereeing system} is crucial for SAGE; otherwise,
a lot of great patches simply won't make it into
SAGE, which will discourage potential developers.
\vfill
\item This referee system will unfortunately slow down the inclusion of
new code into SAGE, but will in the long run {\dred raise
quality} and {\dred scales well}.
\vfill
\item It will also help improve the chances work
on SAGE will be viewed positively by {\dred hiring and fellowship
committees}.
\end{itemize}
\end{frame}
\begin{frame}
\frametitle{SAGE Patch Referee System}
\begin{itemize}
\item I would officially like the following people
to become the first group of patch referees:
{\dgreen Martin Albrecht, Nick Alexander,
Robert Bradshaw}, and {\dgreen David Harvey.}
\item Proposed Workflow (a nag
programs is needed for this):
\begin{enumerate}
\item I receive a patch. I {\dred send it} to one of the
referees.
\item The referee applies the patch, reads through
all relevant code added in the patch, and adds {\dred comments
specifically asking} for more doctests, asking for
explanations about how/why things are coded in a
certain way, asks for coding conventions to be met, etc.
\item The {\dred referee records as a new patch} the commented
code.
\item The {\dred patch author answers} in
the form of another patch.
\item The above three steps are iterated until the
{\dred referee is happy}.
\item I receive the patch, read it, and apply it
unless there are still problems.
\end{enumerate}
\item This process would {\em not} be anonymous.
\item Contributors would be strongly encourage to list
refereed patch contributions as such in a section
of their CV's.
\end{itemize}
\end{frame}
\begin{frame}
\frametitle{Funding}
Funding for SAGE is {\dred crucial}. {\dred Why??}
\begin{itemize}
\item So faculty and graduate student can {\dred work on SAGE more} and
teach less (or do teaching that is more related to either
SAGE or their research).
\item So {\dred undergraduates can work on mathematics-related
software} as a job instead of working for other departments or
companies. (E.g., the UW undergrads working on
SAGE all had jobs in web page design and biology
support before working on SAGE). This is {\dred better for
their education}.
\item To fund {\dred SAGE Days workshops} so developers and users
can meet.
\item The competition (Magma, Maple, Mathematica, and Matlab)
{\dred has many many millions of dollars} per year in funding, and it's
difficult to even begin to compete without funding.
\end{itemize}
\end{frame}
\begin{frame}[fragile]
Funding SAGE isn't about {\dred me} getting grants, it's about
{\dred everyone} getting grants. Undergrads who apply to university
research programs for stipends, grad students who apply to local
institutes for workshops, etc. These are all happening. Keep it
up!
\begin{itemize}
\item I still have some startup money left -- enough
to keep things going for at least 1 more full year,
and possibly 2 years, even if I get no funding
I've applied for.
\item GOOD NEWS: Email from Jon Hanke yesterday
{\small
\begin{verbatim}
The good news is that I have [substantial] startup
money, which I'd like to purchase a reasonably
sized computer to do (numerics and) SAGE development,
hire student programmers, and possibly fund 2 SAGE
Days conferences at UGA (if you'd like). So if you
have any thoughts about how you'd like this to work,
I'd be happy to try to implement them. =)
\end{verbatim}}
\item SAGE Days 4? Probably in late August at UW??
\item Possibilities for SAGE Days 5 -- CRM in Vancouver
(Nils Bruin); Leiden in Netherlands.
\end{itemize}
\end{frame}
\section{SAGE: Technical Status Report}
\begin{frame}
\frametitle{SAGE: Technical Status Report}
\vfill
\begin{center}
{\dred \LARGE SAGE 2.1.3.1: Technical Status Report}
\end{center}
\vfill
\end{frame}
\begin{frame}[fragile]
\frametitle{The Components of SAGE: What and Why}
\begin{itemize}
\item cddlib-094b -- program to generating all vertices and extreme rays of a general convex polyhedron in $\mathbf{R}^d$ given by a system of linear inequalities (needed by gfan, but I wish it's functionality were directly exposed).
\item clisp-2.41 -- the common lisp interpreter (needed by Maxima; could be replaced by GCL, but clisp is the only {\em only} lisp that {\em really} builds everywhere)
\item conway\_polynomials-0.1 -- a database of polynomials that define finite fields (not really used -- should be used to implement natural embeddings between finite fields -- volunteers?)
\item cremona\_mini-0.1 -- Cremona's database of elliptic curves of conductor up to $10000$.
\item doc-2.1.3 -- install guide (good), tutorial (good, but needs update), reference manual (needs way more explanatory overviews and better organization), and constructions guide (needs to be updated!)
\end{itemize}
\end{frame}
\begin{frame}
\begin{itemize}
\item ecm-6.1.2 -- elliptic curve factorization method (fast -- good dsage example)
\item examples-2.1.3 -- examples of use of SAGE (needs to be revamped so all examples can be automatically tested -- this would be a good coding sprint project).
\item extcode-2.1.3 -- code used by SAGE in other languages (e.g., in GP/PARI).
\item flintqs-20070102 -- Bill Hart's multi-polynomial quadratic sieve.
\item freetype-2.1.10 -- used by GD
\item gap-4.4.9 -- Group theory.
\item gd-2.0.33.p4 -- fast 2d graphics library (render, e.g., fractals much more quickly than with matplotlib).
\end{itemize}
\end{frame}
\begin{frame}
\begin{itemize}
\item gdmodule-0.56.p2 -- python bindings for gd library
\item genus2reduction-0.3 -- conductor and reduction type of genus 2 curves (an important orphaned C program)
\item gfan-0.2.2.p1 -- compute with Groebner fans; lies at the heart of the popular research area called ``tropical goemetry''.
\item givaro-3.2.6 -- fast finite field arithmetic library
\item gmp-4.2.1.p4 -- arbitrary precision integers and rationals
\item gsl-1.8 -- the GNU Scientific Library; a mature self-contained C library for a huge range of numerical computation (discuss how GSL has similar functionality to scipy)
\item ipython-20061028 -- the interactive IPython command line (provides SAGE's interactive shell)
\item ipython1-20070130 -- makes it easy to interactively control execution of dozens of copies of SAGE on a cluster
\end{itemize}
\end{frame}
\begin{frame}
\begin{itemize}
\item lcalc-20070107 -- Mike Rubinstein's C++ program for computing with L-functions; it has tons of functionality that hasn't yet been exposed in SAGE.
\item libpng-1.2.8.p0 -- library for manipulating png images.
\item linbox-20070214 -- a very powerful C++ library for exact linear algebra
\item matplotlib-0.87.7 -- 2d Python graphics package; provides Matlab like functionality, and -- thanks to Alex Clemesha -- SAGE provides Mathematica-like 2d plotting on top of matplotlib.
\item maxima-5.11.0 -- a very mature sophisticated computer algebra
system for symbolic computation. I am honestly worried about
efficiency and maintenance issues involved with building SAGE's CAS
functionality on top of Maxima, since CAS is very important for the
mission of SAGE. We shall see.
\end{itemize}
\end{frame}
\begin{frame}
\begin{itemize}
\item mercurial-0.9.3.p1 -- distributed revision control system; every single
copy of SAGE (both built-from source or binary) contains 5 separate
mercurial repositories! Everyone can check in changes, send patches
around, etc. This has been crucial to building the developer community.
\item moin-1.5.6 -- a mature full-functional wiki is included in SAGE.
\item mpfi-1.3.4-rc3.p2 -- very fast arbitrary precision interval arithmetic
\item mpfr-2.2.1 -- we build fast arbitrary precision real and
complex number arithmetic on top of MPFR
\item mwrank-20061123 -- John Cremona's program for computing with
elliptic curves. There is some functionality in this program that
could be made available in SAGE that isn't currently available.
\item networkx-0.33 -- a solid graph theory package; Emily Kirkman
and Robert Miller have done substantial work to make this much
more usable from SAGE.
\end{itemize}
\end{frame}
\begin{frame}
\begin{itemize}
\item ntl-5.4.1 -- fast arithmetic in many cases, though it frequently
gets beat by MAGMA. Required by both Singular and Linbox.
It has a good LLL implementation. The SAGE/NTL interface needs
to be optimized (recent work of David Harvey).
\item numpy-1.0.1.p1 -- excellent library for
numerical linear algebra in Python; needs to be better documented
and easier to use from SAGE. Very powerful. Is core infrastructure
for the Python numerical computation community (which is much bigger
than the SAGE community!)
\item openssl-0.9.8d.p1 -- library for
security internet communication; currently used by dsage, and will
soon be used by the SAGE notebook.
\end{itemize}
\end{frame}
\begin{frame}
\begin{itemize}
\item palp-1.1 -- a program for computing with
lattice polytopes, which are objects of great importance
in algebraic geometry, combinatorics, number theory, etc.
\item pari-2.3.1.cvs-20061215 -- free number theory library
and interactive calculator; much of SAGE's number theory functionality
is built on top of PARI. Also, the PARI library provides
arbitrary precision
numerical integration, evaluation of special functions to
high precision, etc.
\item pexpect-2.0 -- used so SAGE can communicate with almost any other
program via a pseudo-tty. (There is a pexpect-2.1, but it doesn't
work for crap with SAGE. Why?!)
\end{itemize}
\end{frame}
\begin{frame}
\begin{itemize}
\item pycrypto-2.0.1 -- package that implements cryptographic
algorithms and protocols. E.g., Hash functions (MD2, MD4, RIPEMD, SHA256),
block encryption algorithms (AES, ARC2, Blowfish, CAST, DES, Triple-DES, IDEA, RC5), stream encryption algorithms (ARC4, simple XOR), and public-key
algorithms: RSA, DSA, ElGamal, qNEW. YES -- quality implementations
of all these are included standard in SAGE. That this is the case,
and how to use them should be documented much better.
\item pyopenssl-0.6 -- needed by Python programs that use openSSL (e.g., Twisted).
\item pyrexembed-0.1.1-20060517 -- currently not used much; slated for possible removal (?)
\item pysqlite-2.3.2 -- allows one to use the sqlite relational database from Python.
\item python-2.5.p4 -- the Python interpreter, which is the interpreter for SAGE.
\end{itemize}
\end{frame}
\begin{frame}
\begin{itemize}
\item readline-5.2 -- distributed with SAGE since the readlines on a lot of systems are often totally screwy, and often people don't have the readline devel headers.
\item sage-2.1.3 -- the core SAGE library; implementation of new algorithms and defines the ``unified interface'' to all these packages that SAGE presents to the world.
\item sage\_c\_lib-2.1.3 -- a range of code used by the SAGE libraries
that is written in C. E.g., our interrupt handler is here, along
with a number of cool optimization tricks that came out of SAGE Days~2.
\item sage\_scripts-2.1.3 -- these are mostly shell scripts that implement
the SAGE package manager.
\item sagex-20070126 -- SageX is a sort of ``compiled variant of Python''. It's
utterly crucial for writing certain types of code for SAGE. Martin Albrecht
will give a great talk about SageX. This is our (unfortunately necessary)
fork of Pyrex.
\end{itemize}
\end{frame}
\begin{frame}
\begin{itemize}
\item singular-3-0-2-20070105 -- polynomial computation. Has a huge
range of functionality for commutative algebra, and is also supposed
to have powerful non-commutative algebra functionality as well.
None of the non-commutative functionality has been made available
directly in SAGE (coding sprint idea). Martin Albrecht has also
embarked on a fascinating project to make Singular directly usable
by SAGE at the C-library level -- which promises to be ``frickin'
insanely fast'' in some ways.
\item sqlite-3.3.11 -- SAGE's relational database: ``SQLite is a small
C library that implements a self-contained, embeddable,
zero-configuration SQL database engine.'' For certain types of
mathematical tables, SQLite will be amazingly useful.
\item sympow-1.018.1.p1 -- Mark Watkins's program for computing symmetric
power $L$-functions. Fairly specialized.
\end{itemize}
\end{frame}
\begin{frame}
\begin{itemize}
\item tachyon-0.97.p1 -- a very lightweight ray tracing program. Fun. Has
motivated development of some useful 3d graphics and plotting algorithms
by Tom Boothby and Josh Kantor.
\item termcap-1.3.1 -- used by many other programs to determine terminal capabilities (friend of readline).
\item twisted-2.5.0.p2 -- ``the ultimate'' event driven networking
framework; support TCP, UDP, SSL/TLS, multicast, Unix sockets, a
large number of protocols (including HTTP, NNTP, IMAP, SSH, IRC,
FTP, and others), and much more. This is a typical example of the
sort of functionality SAGE has that sets it apart from
Maple/Mathematica/Magma/Matlab.
\item twistedweb2-0.2.0 -- relevant for Twisted and the SAGE Notebook, but
not used by SAGE yet.
\end{itemize}
\end{frame}
\begin{frame}
\begin{itemize}
\item weave-0.4.9 -- mature system from inlining pure C code inside Python
programs. Very powerful. I've never used it, but Josh Kantor has and swears
by it. It's especially good for numerical computation.
\item zlib-1.2.3.p1 -- compression.
\item zodb3-3.6.0.p1 -- a robust mature Python object-oriented
database; basically this is a way to have a potentially huge Python
dictionary that is mostly stored on disk.
\end{itemize}
\end{frame}
\begin{frame}
\frametitle{Main Current Development Directions}
For SAGE, I think this is the year of {\em optimization}.
\begin{itemize}
\item {\dred Parallel computation} -- {\dgreen an optimization technique}:
\begin{enumerate}
\item Crucially requires robust {\dred object serialization} (pickling).
\item {\dred DSAGE} -- Goal is super-easy task farming.
\item {\dred IPython} -- Goal is easy control of a bunch of SAGE's at once (and
support for using MPI).
\item {\dred pthreads} -- Limited use for low level libraries
\end{enumerate}
\item {\dred Fast numerical and exact linear algebra}: use and improve
linbox. Implement new advanced optimized algorithms in the context
of SAGE (in progress). More optimized classes.
\item {\dred Fast basic arithmetic:} FLINT, optimize all basic
arithmetic (e.g., SageX more of polynomials, power series,
$p$-adics, etc), fast multivariate polynomials via Martin's new
Singular library interface, speed up object creation and memory
management.
\end{itemize}
\end{frame}
\begin{frame}
\frametitle{Discussion}
\vfill
\begin{center}
{\dred \LARGE Discussion}
\end{center}
\vfill
\end{frame}
\end{document}
\frametitle{Linear Algebra: Status}
{\dred Goal:} We need modern fast optimized implementations
of algorithms for linear algebra over $\Z$, $\Q$, $\F_q$, $\Q_p$, number fields,
certain polynomial rings, and double-precision real and
complex fields. {\dblue Both dense and sparse with
identical functionality.}
{\dred Status:} The pieces are in place but the
current linear algebra implementation is
often embarrassingly slow. Needs systematic
work.
\begin{itemize}
\item (done) {\dred Initial Python implementation} so what functionality is needed is clear, and a package
that {\em uses} linear algebra in a highly nontrivial way (namely modular symbols).
\item (done) {\dred Rewrite using SageX} -- compiled language, and restructure implementation to make
it possible to systematically create optimized matrix classes for different base rings.
\item {\dred Incorporate linbox and IML} into SAGE (in progress); fix bugs, optimize, discover gaps, etc.
IML is GPL'd but Linbox is LGPL'd (so I am reluctant
to give back my work to Linbox).
\item {\dred Implement advanced optimized algorithms} in the context of SAGE (in progress);
fix bugs, etc.
\end{itemize}
\end{frame}