Equatorial Mathematics

Experimental Sage-based Mathematics course

Dima — Sun, 07 Aug 2011 08:28:56 +0000

We are to embark upon teaching a 2nd year undergraduate course Experimental Mathematics, which will cover computer algebra basics, and refresh concepts from 1st year linear algebra, calculus, and combinatoris, with few more advanced things thrown in. The course will be based on Sage, with the actual software running on dedicated servers, and accessible via Sage web notebook interface (i.e. basically nothing but a web browser running on the student’s computer/laptop/ipad, etc).

Given the enrollment of about 120, this will be interesting…

PS. Most students did not appreciate the freedom given, and complained, complained, complained…

An email from Sendai (Tohoku University)

Dima — Sun, 13 Mar 2011 15:30:08 +0000

I was very glad today to receive an email from Akihiro Munemasa (from Tohoku university at Sendai), my long-term collaborator and friend. He says he’s OK, but has no electricity/water/gas…

And, as we see, they have internet, at least some kind of service (email went via me.com, a web-mail service run by Apple).

Well, I can only wish a lot of strength to Akihiro, and all the people affected by this disaster!

PS: today (14th March) he also sent me photos of his office:

2010 in review – Happy New Year!

Dima — Sun, 02 Jan 2011 14:01:27 +0000

While it might be snowing on this page, Edith picked up a mango fallen from a tree outside our block of flats this afternoon…

The stats helper monkeys at WordPress.com mulled over how this blog did in 2010, and here’s a high level summary of its overall blog health:

The Blog-Health-o-Meter™ reads Fresher than ever.

Crunchy numbers

A helper monkey made this abstract painting, inspired by your stats.

A Boeing 747-400 passenger jet can hold 416 passengers. This blog was viewed about 5,700 times in 2010. That’s about 14 full 747s.

In 2010, there were 6 new posts, growing the total archive of this blog to 33 posts. There were 5 pictures uploaded, taking up a total of 28kb.

The busiest day of the year was May 11th with 79 views. The most popular post that day was Is C++ the worst 1st programming language for maths majors?.

Where did they come from?

The top referring sites in 2010 were terrytao.wordpress.com, cameroncounts.wordpress.com, debian-news.net, edventure.ntu.edu.sg, and symomega.wordpress.com.

Some visitors came searching, mostly for funny pictures, funny, confused cat, total unimodularity, and funny pics.

Attractions in 2010

These are the posts and pages that got the most views in 2010.

Is C++ the worst 1st programming language for maths majors? May 2010
4 comments

Total unimodularity and networks October 2009
1 comment

Cosets and normal subgroups March 2009

Bipartite matchings via linear programming September 2009
1 comment

Bellman-Ford algorithm for shortest paths and potentials September 2009

Python extensions with C libraries made easy by Cython

Dima — Sat, 16 Oct 2010 07:19:40 +0000

Suppose you have a library written in C that you like to be called from Python. There are many ways to accomplish this, and I would like to show here a complete example of doing this using Cython, which is basically a compiler from (an extension of Python) to C. Most of this, and much more, can be found in a quick tutorial.
For the purpose of an example, suppose you have a C function that computes the average of an array of doubles (all source files here, as well as the Makefile that basically contains the shell commands we show below, can be downloaded here):
/* cmean.c */ double mean(int n, double* a) { double s; int i; for (s=0., i=0; i return s/n; }
with the prototype
/* cmean.h */
double mean(int, double*);

and you want to call it from Python, i.e. (we need to give the function another name here, cmean(), for technical as well as for semantic reasons---noone would pass the length of an array and the array to a function in Python, as the array itself would certainly do) say
b=[1.3,5.777777,-12.0,77.] print cmean(b)
would print the same as
print sum(b)/len(b)
As cmean() is not a Python built-in, we would expect to import it first. So the following
$ python test.py
where
# test.py import dyn from dyn import cmean b=[1.3,5.777777,-12.0,77.] print cmean(b) print sum(b)/len(b)
will print the number 18.01944425 twice. OK, so here is cmean() definition in Cython (note the file extension .pyx)
# m.pyx cdef extern from "cmean.h": double mean(int, double*) from stdlib cimport * def cmean(a): n = len(a) cdef double *v v = malloc(n*sizeof(double)) for i in range(n): v[i] = float(a[i]) m = mean(n, v) free(v) return m
The main task of cmean() is to create a "plain" C array of doubles, pass it to our C function, get the result, clean after itself, and return the result. Unlike Python, Cython has types declarations. Unless one calls C-function from the Cython code, they are not mandatory though, although they can speed the things up tremendously in "real" computations. In the example here
cdef double *v
cannot be avoided --- the code does not compile without it. In a usual C-like fasion, instead of 2 lines
cdef double *v v = malloc(n*sizeof(double))
one could have written just
cdef double *v = malloc(n*sizeof(double))
The top two lines pass the prototype declaration of the C-function mean to be included into the C file to be generated, and the third line imports the functions from the C library stdlib (we need malloc and free from there).

Having this, we need to create the extension module dyn, to be imported by Python. This is perhaps the least intuitive part of the job. One way to accomplish it is to use Python distutils: i.e. we need to create a setup.py file to be called as
$ python setup.py build_ext --inplace
This should look as follows:
# setup.py from distutils.core import setup from distutils.extension import Extension from Cython.Distutils import build_ext ext_modules=[ Extension("dyn", ["m.pyx"], library_dirs = ['.'], libraries=["cmean"]) # Unix-like specific ] setup( name = "Demos", cmdclass = {"build_ext": build_ext}, ext_modules = ext_modules )
One not yet explained thing here is how the shared library containing the compiled C-function mean() is hooked up to dyn. You see in the code above the declarations library_dirs and libraries. They assume that the shared library is called libcmean.so and that it is in the same directory (i.e. '.' is the path to it) as the rest of the code. One can create libcmean.so (before creating dyn, more precisely, a file called dyn.so) by running
$ gcc -fPIC -shared cmean.c -o libcmean.so
One more point to watch is that on some Unix systems the loader will be unable to locate the dynamic libraries libcmean.so and/or dyn.so we created (unless they are placed in certain stanadard directories, which is not something one wants to do with experimental code!). So test.py will need to be run, e.g., as follows:
$ LD_LIBRARY_PATH=. python test.py

For the sake of completeness, the aforementioned archive contains a C file that calls mean from libcmean.so, and the corresponding entries in the Makefile can do the necessary work to build and run it.

Last but not least, Sage automates parts of the interface building even better.

Integer flows and graph theory applications

Dima — Thu, 14 Oct 2010 04:58:50 +0000

Here we are continuing the story of Ford-Fulkerson algorithm. Assume that , i.e. the capacities are all integers. Then starting the Ford-Fulkerson from the zero flow, we see that at each iteration . Thus the optimal flow will have integer values for each arc (such flows are called integral). We have

Corollary 3 A maxflow problem with integer capacities has an integral optimal solution.

This has various graph-theoretic applications. We can convert an undirected graph into a digraph by replacing each edge by a pair of opposite arcs and , set all arc capacities to 1, and see what this implies for a pair of non-adjacent vertices . Running Ford-Fulkerson algorithm, we obtain an integral maximum flow , of value .

We also know that there is, by the maxflow-mincut Theorem, an cut s.t. . Instead of capacity, it is more appropriate to talk here about the size of the cut, i.e. the number of edges crossing from one part of the cut to another. Obviously, by definition of the capacity, we have .

The flow can be viewed as a collection of edge-disjoint -paths from to , as can be seen by induction on : indeed, if then by construction (via the Ford-Fulkerson algorithm) is a path. Suppose now this statement holds for all . As induces a connected subgraph, it contains a shortest path . Then, is cut by a minimum cut —otherwise the cut does not cut! Removing all the edges of from , we thus obtain a graph that has the cut of size , for 0}' title='{\ell>0}' class='latex' />. The flow in corresponding to has value , and is an augmenting path for this flow, with . Thus , i.e. cuts just 1 edge in , and by induction contains edge-disjoint -paths, and we obtain

Corollary 4 (Menger Theorem (for edge-disjoint paths)) The minimum size of an cut in equals the maximal number of edge-disjoint paths in .

There is a graph transformation that allows one to prove Menger Theorem for internally disjoint paths (two paths between and are called internally disjoint is their only common vertices are and ). Our goal is to prove the following.

Theorem 5 (Menger Theorem (for internally disjoint paths)) The minimum size of an cut in equals the maximal number of internally disjoint paths in .

Proof: Assume and are not adjacent. Replace each by a pair of vertices , , joined by an arc . Then replace each edge of by a pair of opposite arcs and . Replace every arc such that by the arc , each arc with or by , and each arc with or by . Now each path corresponds to the path in the new graph, and the opposite holds, too: each path in the new graph has this form.

Assign capacities 1 to the arcs , and infinite capacities (it suffices to take this “infinity” to be bigger than, say, ) to the rest of the arcs. Due to the choice of capacities, a minimum cut will satisfy , and thus the only arcs from to will be of type . Say, there will be such arcs. By (a straightforward generalisation of) Menger Theorem for edge-disjoint paths (to directed graphs), there will be exactly arc-disjoint paths in this digraph. By its construction, they cannot share a vertex (resp. ), as there is only one outgoing (resp. incoming) arc on it. Thus these paths are internally disjoint, and the corresponding paths in are internally disjoint, too.

One more classical application is a maxflow-based algorithm to construct a maximum matching in a bipartite graph , with the bipartition . We add two more vertices and to it, and connect to each by an edge. In the resulting new graph we solve the maxflow problem with all capacities set to 1. More precisely, we orient each edge so that each , , becomes an arc , respectively, , respectively .

Ford-Fulkerson algorithm, when started from the zero flow, will produce an integral maximal flow, of value . As we know from Menger’s theorem, we will have internally disjoint -paths. They will induce a matching on (just look at the edges only). On the other hand, any matching on can be used to construct an -flow of value in with all capacities 1. Hence is a maximum matching in .

Augmenting paths for Maxflow and Mincut

Dima — Sun, 10 Oct 2010 16:17:09 +0000

We discussed how to formulate the MAXFLOW problem for networks, i.e. arc-weighted digraphs with two specific vertices, source and sink and arc capacities , and treat it via linear programming. An alternative approach is via augmenting paths. The paths we talk about will disregard the arc directions in , i.e. they will be - paths in the underlying undirected graph. (When has the direction opposite to the arc in , we write .)

Definition 1 Given an - flow , an -augmenting path is an - path in the underlying graph, such that for each

if then ;
if then 0}' title='{f(e)>0}' class='latex' />.

The tolerance of is

Note that the definition of tolerance extends unchanged to any path starting at .

We can use to improve , as follows.

Lemma 2 For each , let

Then is an - flow in , of value greater than that of by

Proof: It is immediate from the definition of that for any . We still need to check the flow conservation constraints

If visits then there are two arcs, say, and on that are affected when we change to , and the changes in flow values cancel each other in each of the four cases (recall that is a path in the underlying undirected graph). Finally, the value of is the net outflow from , and the latter is greater than that of by .

This suggests finding a maximum flow by repeatedly finding an augmenting path and improving the current flow with it, till no augmenting path is available (this is, for instance, how Ford-Fulkerson algorithm is working). For this to succeed, we obviously need to prove that is maximum if and only if it does not have an -augmenting path.

Augmenting paths are found by a breadth-first search from . At some point, if we did not succeed in reaching , we would end up with the set of vertices reachable, in the underlying non-oriented graph, by paths with positive tolerance. Each crossing over from to satisfies the property that , and each arc crossing from to satisfies — otherwise we would have added their vertices in to . Therefore,

where on the right we have the definition of capacity of an arbitrary - cut .

It is easy to show that for an arbitrary - cut , one has . Thus we have have obtained the Maxflow-Mincut Theorem, and also proved that our algorithm, to be described in more detail shortly, finds a maximum flow , and a corresponding to it minimum cut .

Breadth-first search: Ford-Fulkerson algorithm

As usual in breadth-first search, we use a queue, i.e. a data structure that has two indices, one, , pointing to the end of the queue, for new arrivals, and the other, , pointing to the beginning of the queue, i.e. to the first element to be processed. There will be elementary operations:

– adds a new element, , to the queue, and icrements . In this case we also do not allow an element to enter the queue twice, so has no effect if already was these once.
– takes the (currently) first element, , from the queue, and icrements —this will fail if is empty, i.e. if .

Now the algorithm is as follows:

Input: a feasible flow in with source and sink .
Initialization: let be an empty queue; , and set .
Loop: take . If this fails, return the first coordinates of the pairs in —a minimum cut.
for s.t. and , or s.t. and 0}' title='{f((vx))>0}' class='latex' />:
- , .
- if then return the augmenting - path by tracing back the pairs , ,\dots, in .
go to Loop.

So if is not maximum we get an -augmenting path, that can be used to improve , and repeat. However, the speed of this procedure becomes unacceptable when the capacities are large. If then we are at least assured that it will converge after finitely many steps: indeed, we can multiply by the least common multiple of its denominators, reducing to the case . Each iteration in this case improves the flow by at least . However, the time would not be polynomial in the input of the problem. E.g. for the following graph the number of iterations can be as large as , when is a large integer.

For irrational the convergence not even need to be the case!

If one chooses a shortest -augmenting path to improve with, then the running time becomes bounded by —that was shown by E.Dinitz in 1970 and independently by J.Edmonds and R.Karp in 1972. Details, and further improvements by A.Karzanov et.al can be found e.g. in A.Schrijver’s lecture notes, Sect.4.4.

Edmonds algorithm for maximum matchings

Dima — Tue, 05 Oct 2010 03:38:53 +0000

We continue with matchings in simple graphs, following the exposition in A.Schrijver’s 3-volume work, and his lecture notes available online, filling in few more details.

While in a bipartite graph and a matching it is always the case that a shortest path from the set of -unsaturated vertices to itself (i.e. an --path) is always -augmenting, this is no longer true in general graphs:

here , is indicated by thicker edges, and the shortest path (of length 4) between them is not augmenting. This precludes the algorithmic idea of augmenting a matching using an augmenting path (found by a shortest path procedure in a digraph constructed from and ) from working in the general case. Edmonds came up with an idea that one can allow for repeated vertices, and shink the arising in the process cycles to (new) vertices.

Definition 1 An -alternating walk is a sequence of vertices (some of which can repeat!) that is a walk (i.e. for all ) so that exactly one of and is in .

Definition 2 An -alternating walk is an -flower if has an even number of vertices (as a sequence, i.e. is odd), , all the are distinct, and for (Removing the vertex and the edge on the picture above is an example of an -flower).
The cycle in an -flower is an -blossom. ( on the picture above is an example of an -blossom.)

Once we have an -blossom , we can shrink it by replacing all of its vertices with just one vertex (call it , too), replacing each edge , with the edge (and ignoring any loops that arise on ). The resulting graph is denoted by . (E.g. shrinking the -blossom on the picture above produces the graph that is the path ). In one will have the matching , where will become just a saturated vertex.

Theorem 3 is a maximum matching in if and only if is a maximum matching in , for an -blossom .

Proof: If is not a maximum matching in then there will be an -augmenting path in If does not meet then it corresponds to an -augmenting path in . So we can assume that is an edge of , and is also an edge of . As , there exists so that Now, depending upon the parity of , we replace in by the part of the blossom rim , where the sign in the indices is for odd, and for even.

Conversely, if is not a maximum matching in , then

is also a matching of the same size, and . Thus it suffices to show that is not maximal in , and we will assume w.l.o.g. that , i.e. (our flower has no stem). There exists an -augmenting path . W.l.o.g. does not start in (we can reverse it if it does). If does not contain a vertex in , it will continue to be an augmenting path in . Otherwise let the first vertex of in be . Then is an augmenting path in .

For the algorithm we will need

Theorem 4 A shortest to -alternating walk is either a path or its initial segment is an -flower.

Proof: Assume is not a path, and let be such that and as small as possible. We will show that is an -flower, and we immediately have it if is even and is odd.

If were even, as shown above, where the walk might have been , and so , , then the path is -alternating, so we would simply delete it from and obtain a shorter -alternating walk.

If were odd and even, we would have and in , so , contradicting the choice of .

Combining these two observations, we obtain the following augmenting path algorithm:

Input: a graph and a matching .
Let be the set of vertices unsaturated by . Construct, if possible, a positive length shortest walk from to .
If no was constructed, return — is maximum.
If is a path then return it.
Otherwise take the first blossom from , call the algorithm recursively on and , and return the expanded to an augmenting path in resulting path in .

Applying the algorithm starting from , and augmenting with the path it returns, we obtain an efficient algorithm, running in fact in operations.

We still need to discuss here how to do Step 2. To this end we construct a directed graph with

Lemma 5 Each to -alternating path, resp. each -flower, corresponds to a directed path in from to —the set of neighbours of in Conversely, each to directed path in corresponds either to an -augmenting path or to an -flower.

Proof: (Sketch) Each length 1 -augmenting path obviously corresponds to the pair of arcs and in .

If is a -path in , so that , , then there is a unique, up to the choice of , lifting of it to an -alternating walk , with for . If is a path then it is -augmenting.

If has repeated vertices, this repetition occurs in a regular way, corresponding to an -flower. As we move along from to , let , i}' title='{t>i}' class='latex' />, be the first repetition. Then is the beginning of the blossom, and the last vertex of the blossom.

Installing CPLEX 12.2 on Debian amd64 and MacOSX 10.6 (64-bit)

Dima — Fri, 24 Sep 2010 12:02:51 +0000

IBM now gives CPLEX to academics for free; as I am computing LP bounds for quantum codes, I got myself one. However, it refused to install on my amd64 Debian system, saying

"libgcc_s.so.1 must be installed for pthread_cancel to work".

After much googling, it turned out that the installer is a 32-bit program, that needs the right 32-bit libgcc_s.so to work.
To get this on Debian (squeeze), I just did
$ apt-get install ia32-libs
Then
$ LD_LIBRARY_PATH=/usr/lib32; sh cplex_studio122.acad.linux-x86.bin
worked as it should.

The installation goes smoothly on a 64-bit MacOSX 10.6. But running is not: one has problems when doing
>>> import cplex ... from cplex._internal.py1013_cplex122 import * ImportError: dlopen(/Library/Python/2.6/site-packages/cplex/_internal/py1013_cplex122.so, 2): no suitable image found. Did find: /Library/Python/2.6/site-packages/cplex/_internal/py1013_cplex122.so: mach-o, but wrong architecture

One can make it work on by setting
$ export VERSIONER_PYTHON_PREFER_32_BIT=yes
before starting the Python session (as IBM/CPLEX Support kindly told me; they promised a fix in an upgrade, too).

Is C++ the worst 1st programming language for maths majors?

Dima — Thu, 06 May 2010 16:50:12 +0000

In our department we have a 1-semester course called “Introduction to scientific programming”, that is taught to the 1st year undergrads. It is C++-based. I am arguing for years that it is a real waste, and only good for getting people dazed and confused, and that instead something like Python should be used.

The choice of Python is basically due to it being interpreted, (largely) untyped, and higher level than C/C++.
(And last but not least, free, unlike Maple, Matlab, and Mathematica, which might be marginally more useful to a maths major).

Here is a ranking of programming languages in industry. Inevitably biased in some ways, I suppose it still demonstrates that claimed “superiority” of C++ is nowhere to be seen…

Total unimodularity and networks

Dima — Mon, 12 Oct 2009 07:27:40 +0000

Total unimodularity is a tool to show that a linear programming problem (LP) has integer optimal solutions. In our discussion on bipartite matchings and linear programming we saw how this is possible to do by hand in a particular case, but as the structure of the LP gets more complicated the details can get very messy. Fortunately, there is a general tool that sometimes gives what is needed.

Definition. A matrix is called totally unimodular if each of its subdeterminants equals 0, 1, or -1. (In particular, each entry of is 0, 1, or -1.)

Proposition. Let be totally unimodular and Then the polyhedron is integral (i.e. each face has an integer point.)

Proof. For a minimal (w.r.t. inclusion) face of one has where resp. is a submatrix, resp. subvector, of resp. of and has full row rank. Then, possibly after permuting coordinates, we have with By total unimodularity, Then and satisfies

To complete the proof we need to check that This is where the minimality of is needed. In the simplest case is a vertex, and implying the claim. In general, i.e. a translate of the nullspace of So one can reduce to the vertex case by adding equations of the form for each column of the submatrix this does not destroy the feasibility of QED.

As a corollary, the LP has integer optimum (and optimiser) whenever Moreover, the same holds for its LP dual by rewriting and applying the proposition above.

Hoffman-Kruskal Theorem. It turns out that this property characterises
total unimodularity:

is totally unimodular iff for any the polyhedron is integral.
is totally unimodular iff for any , the equality in the LP duality equation is achieved by integral vectors (assuming these min and max are finite).

The second obviously follows from the first.
The first will follow from the following:

A full rank matrix is totally unimodular iff for any the polyhedron is integral.

We omit the proof here.

Examples of totally unimodular matrices.

(Vertex-edge) incidence matrix of a graph is totally unimodular iff is bipartite.

Sketch of the proof.
1) Prove that the incidence matrix of an odd cycle is not totally unimodular, e.g. by considering the vertex numbering such that the edges are and arranging the columns of the matrix in this order, while arranging the rows in the order Then the usual decomposition of the determinant w.r.t. all the permutations of symbols will have only 2 nonzero summands, that will not cancel each other (actually, in the case of even they will cancel each other).
This shows that cannot be totally unimodular if is not bipartite.
2) Let be a square submatrix of If it has a 0 column its determinant is 0. If it has a column with just one 1, we can decompose the determinant w.r.t. this column and apply induction on the size of This leaves one with the case of having exactly two 1s per column. Same argument can be applied to argue that has, w.l.o.g., at least two 1s per row, and thus exactly two 1s, by counting 1s of in two ways. It follows that is the incidence matrix of a disjoint union of (even, as is bipartite) cycles. By reordering the rows and columns we and make to be block-diagonal, with each block the incidence matrix of an even cycle. The determinant of a block-diagonal matrix equals the product of determinants of the blocks, and so it remains to show that the incidence matrix of an even cycle is totally unimodular; this can be done as above. Q.E.D.

Excercise. Combine this with the result on the total unimodularity above to show that the maximum matching LP for bipartite graphs has an integer solution. Construct the LP dual for this problem and interpret its integer solutions (show that they exist first!) in graph-theoretic terms.

Excercise. The incidence matrix of a digraph is a 0,1,-1 matrix of size so that if, respectively, misses or arrives to or leaves Show that is totally unimodular.

Network flows and max-flow min-cut theorem.
Armed with the above, we can easily prove the max-flow min-cut theorem for networks. Recall that a network is a directed graph with two marked nodes and arc capacities An -flow on the network is a function satisfying for all and for any where and (The latter is called conservation law.) The value of is i.e. the “amount” of that “leaves”
And cut is a partition of into two parts and so that The capacity of the cut is the sum of the capacities of the arc leaving

Max-flow min-cut theorem. Given the network the maximum value of an -flow is equal to the minimum capacity of an -cut.

In order to formulate this as an LP, we introduce the function so that if, respectively, misses or resp. enters or resp. leaves Now we can formulate the max-flow problem as follows:

Let be the sumbatrix of the incidence matrix of obtained by removing the rows with indices and Then the latter can be rewritten as

The matrix of this LP is totally unimodular. If then we can apply the integrality result directly, and establish along the way that there exists an optimal integral flow. In general, we cannot apply the unimodularity directly. But we can take the LP dual. It is as follows:

Here the minimum is attained by integer vectors, as the polytope describing the feasible set is integral. We can extend by defining Then The set satisfies So it is a cut. To show the claim of the theorem, it suffices to show that the total capacity of the arcs leaving is bounded from above (the opposite direction is easy) by Let is such an arc. Then and Then implies so Q.E.D.