I think the articles and books below are the most important references for learning about modular abelian varieties. Click on the link for more information about each book.

- Ribet-Stein, Lectures on Serre's Conjectures
- Diamond and Im, Modular forms and modular curves (in particular, Part I, Part II Sections 7, 9.1, 9.2, 10.1)
- Darmon, Diamond, Taylor, Fermat's Last Theorem
- Knapp, Elliptic curves
- Lang, Introduction to Modular Forms
- Manin, Parabolic points and zeta functions of modular curves
- Miyake, Modular forms
- Ribet, Galois Representations and Modular Forms
- Serre, A course in arithmetic (chapter 7)
- Shimura, Introduction to the Arithmetic Theory of Automorphic Functions

- Lang, Introduction to Algebraic and Abelian Functions
- Lang, Abelian Varieties (second edition)
- Milne, Abelian Varieties (mainly Sections 1, 2, 7, 8, 9, 10, 11, 12, 16, 19)
- Milne, Jacobian Varieties (Sections 1-6 and 10)
- Mumford. Abelian Varieties
- Mumford, Curves and their Jacobians
- Rosen, Abelian Varieties over
**C** - Swinnerton-Dyer, Analytic theory of abelian varieties

- Murty, Modular Elliptic Curves (Sections 1-4)
- Ribet-Stein, Lectures on Serre’s Conjecture
- Ribet, Galois Representations and Modular Forms
- Shimura, Introduction to the Arithmetic Theory of Automorphic Forms
- Shimura, On the factors of the jacobian of a modular function field

- Artin, Neron Models (Sections 1 and 2), in Cornell-Silverman
- Birch, Conjectures Concerning Elliptic Curves
- Birch, Elliptic curves over
**Q**, A Progress Report - Kolyvagin, Bounding Selmer Groups via the Theory of Euler Systems (Section 0 only)
- Lang, Number Theory III
- Rubin, The work of Kolyvagin on the arithmetic of elliptic curves
- Tate, The Birch and Swinnerton-Dyer Conjecture and a Geometric Analogue
- Swinnerton-Dyer, The Conjectures of Birch and Swinnerton-Dyer, and of Tate
- Silverman,The Theory of Height Functions (Sections 1-5), in Cornell-Silverman
- Wiles, The Birch and Swinnerton-Dyer Conjecture (for Clay Math Institute)

- Cremona, Algorithms for Modular Elliptic Curves, 2nd edition
- Flynn, Leprevost, Schaefer, Stein, Stoll and Wetherell, Empirical evidence for the Birch and Swinnerton-Dyer conjectures for modular Jacobians of genus 2 curves
- Merel, Universal Fourier Expansions of Modular Forms
- Stein, An introduction to computing modular forms using modular symbols
- Stein, Studying the Birch and Swinnerton-Dyer Conjecture for Modular Abelian Varieties Using MAGMA
- Stein, MAGMA documentation for modular forms and modular symbols

- Atiyah-MacDonald, Commutative Algebra
- Wiles, Modular Elliptic Curves and Fermat's Last Theorem
- Cassels-Frohlich, Algebraic Number Theory
- Hartshorne, Algebraic Geometry