==================
SageMath in Debian
==================

Build instructions
==================

See https://wiki.debian.org/DebianScience/Sage for instructions on setting your
system up to work with this source package.

It's recommended that you build with "DEB_BUILD_PROFILES=pkg.sagemath.ccache".
This uses debian/ccache as a ccache cache, so rebuilds are much quicker.

Local system patches
--------------------

No local system changes should be needed at this time for the main and dochtml
builds to work automatically from a clean check-out.

Remaining tasks
===============

See TODO and https://wiki.debian.org/DebianScience/Sage

Running doctests
================

(These instructions are for running doctests from a built source tree. For
running doctests from an installed binary package, see README.Debian.)

The tests write logging output to sage/logs/ptestlong.log, or some other file
if you ran a different test. If you let the test run to completion, near the
end of the logging output, there will be a summary of all the failures.

After a successful build, you can run individual tests as follows. First, make
sure you're in the top-level package directory, where debian/ is. Then:

 $ . <(debian/rules export-build-env)  # assumes you're in bash or zsh
 sage$ ./sage -t -p --long src/sage/file/you/want/to/test

To run all the tests:

 sage$ ./sage -t -p --all --long --logfile=logs/ptestlong.log

See https://www.sagemath.org/git-developer-guide/doctesting.html for more
information and helpful tips.

Rewinding the build
-------------------

For various technical reasons, you might need to do some extra manual steps for
the above and below commands to work.

- To run them after `dh_auto_install` has succeeded, first move or symlink
  debian/tmp to debian/build.

- To run them after `dpkg-buildpackage` has succeeded, you'll need to apply all
  the debian patches again, then also do the previous point.

Debugging test failures
-----------------------

Always focus on the *first failure* in a given test file. In many cases,
subsequent failures (such as NameErrors) are due to the first failure, and many
of those will go away after fixing the first one.

We have some test-analysis rules that will print you some nice stats about the
failures. For example:

 $ debian/rules failed-tests-by-count
 $ debian/rules failed-tests-by-error
 $ debian/rules failed-tests-by-cause

Look in debian/rules for more. These are useful if you have hundreds or
thousands of failures to deal with and don't know where to begin.

Fixing test failures
--------------------

You should get to know the build process (in debian/rules), so that you can
make fixes to particular test files without having to rebuild everything again,
which takes ages!

There are three cases:

- Fixing a doctest. That is, something that looks like this:

  [.. sage python/cython code ..]
  """Some documentation

  sage: (code to be tested)
  (expected output)
  """
  [.. sage python/cython code ..]

  This is the easiest case - simply edit the file in-place in sage/src, then
  re-run the tests as described above.

- Fixing python code. That is, code in a .py file that is not a doctest.

  Edit the file in-place in sage/src/a/b/etc, then copy it over to
  debian/build/usr/lib/python2.7/dist-packages/a/b/etc (or debian/tmp if
  `dh_auto_install` had previously succeeded) then run the tests again.

- Fixing cython code. That is, code in a .pyx file that is not a doctest.

  You'll need to re-run the build again. First, make sure you've followed any
  applicable commands from the "Rewinding the build" section above. Then:

  $ debian/rules override_dh_auto_build-arch

  This *should* only build the changed files. However we explicitly avoid
  rebuilding the documentation, because incremental docbuild is currently
  broken in upstream (#21612).

  If you are impatient, you can even cancel the build early using Ctrl-C -
  after it copies .so files into debian/build - then run your test again.

You may also find this useful:

 $ debian/rules check-failed

It will run all previously-failed tests again, which is useful if you're on a
mass test-fixing spree and forget to count which ones you've fixed. However
this assumes that you didn't break any of the previously successful ones :p

Finally, after you are satisfied with your fixes, you should do a completely
clean build to verify these fixes - to make sure that they weren't affected by
temporary intermediate build products, e.g. due to our short-cuts above.

Current test status
-------------------

You should get something roughly like the following. If you get different
results, please add your experience to the below summary.

An overview over test failures on different architectures on buildds can be found at
https://people.debian.org/~thansen/sage-test-status.html

You can see infinity0's test failures here: https://people.debian.org/~infinity0/sage/
Look for files named "sagemath-*_*.log"

Known causes::

sage -t --long src/sage/misc/sagedoc.py  # 7 doctests failed // obviously fails when the documentation is not built

Unknown::

sage -t --long src/sage/geometry/polyhedron/backend_cdd.py  # 1 doctest failed

Matplotlib version mismatch (see https://trac.sagemath.org/ticket/23696):

sage -t --long src/sage/plot/arrow.py  # 1 doctest failed
sage -t --long src/sage/plot/plot.py  # 1 doctest failed

Sympy (see https://trac.sagemath.org/ticket/23496,
       maybe fixed with sympy > 1.1.1 by https://github.com/sympy/sympy/pull/12826)

sage -t --long src/sage/interfaces/sympy.py  # 1 doctest failed

GLPK verbosity patch:

sage -t --long src/sage/numerical/backends/glpk_exact_backend.pyx  # 1 doctest failed
sage -t --long src/sage/numerical/backends/generic_backend.pyx  # 1 doctest failed
sage -t --long src/sage/numerical/backends/glpk_backend.pyx  # 4 doctests failed

This patch needs to be upstreamed to GLPK:
https://git.sagemath.org/sage.git/tree/build/pkgs/glpk/patches/glp_exact_verbosity.patch?h=develop
More discussion in the second half of:
https://lists.alioth.debian.org/pipermail/debian-science-sagemath/2017-October/001086.html


Manual testing
==============

Pre-install jupyter notebook tests
----------------------------------

To test the jupyter notebook without installing anything, first see "rewinding
the build" above". Then:

1. Copy debian/build/usr/share/jupyter/kernels/sagemath to /usr/share/jupyter/kernels/
2. Fix up the .svg symlinks there.
3. Edit /usr/share/jupyter/kernels/sagemath/kernel.json to say "./sage" instead of /usr/bin/sage.
4. `./sage -n jupyter`

Examples
--------

These can be tested from the Sage CLI, SageNB, and the Jupyter Notebook.

Typesetting::

    show(integrate(1/(1+x^4), x))

    You should get a long expression with arctans, logs, and square roots.

3D plots::

    def f(x,y):
        return math.sin(y*y+x*x)/math.sqrt(x*x+y*y+.0001)
    P = plot3d(f,(-3,3),(-3,3), adaptive=True, color=rainbow(60, 'rgbtuple'), max_bend=.1, max_depth=15)
    P.show()

    You should get a 3D plot, or a notice about JSmol being unavailable.
