Teaching Fall 2024 Math6350

Deep learning theory with applications

Semester: Fall 2024;

Time: 08/22/2024-12/09/2024, Wed 07:10pm-09:40pm;

Location: GOV 101;

Instructor: Yanxiang Zhao, Phillips Hall 702

Phone: 202-994-0606

Email: yxzhao at email dot gwu dot edu

Office Hour: Mon&Wed 02:00pm--03:00pm or by appointment

Course Description

This course is aimed at advanced undergraduate and beginning graduate students in mathematics, science and engineering. The goal of this course is to provide a broad entry point to applied data science for students with a basic knowledge in linear algebra, differential equations, scientific computing who are seeking to expand their background to include methods in data science, optimization, dynamical systems and control theory.

Course Recording (University policies)

Use of Electronic Course Materials and Class Recordings:
Students are encouraged to use electronic course materials, including recorded class sessions, for private personal use in connection with their academic program of study. Electronic course materials and recorded class sessions should not be shared or used for non-course related purposes unless express permission has been granted by the instructor. Students who impermissibly share any electronic course materials are subject to discipline under the Student Code of Conduct. Please contact the instructor if you have questions regarding what constitutes permissible or impermissible use of electronic course materials and/or recorded class sessions. Please contact Disability Support Services if you have questions or need assistance in accessing electronic course materials.

The course recordings can be found on Blackboard under the tab of 'Zoom meeting'

Prerequisites

• Math 3342: Introduction to Ordinary Differential Equations;
• Math 3553: Introduction to Numerical Analysis (or their equivalents or permission of instructor);
• Knowledge of a programing language.

Textbook

• Generative Deep Learning: Teaching Machines to Paint, Write, Compose, and Play, by David Foster, Oreilly, 2nd edition, 2023;
• Data-Driven Science and Engineering: Machine Learning, Dynamical Systems and Control by Steven L. Brunton and J. Nathan Kutz [link];
• Other useful references:
  1. Geometric data analysis, beyond convolutions [link];
     • This one is advanced book, with many applications in data science.
  2. Mathematical foundations for data analysis [link];
     • This is an undergraduate level textbook with lecture videos.
  3. Machine Learning [link];
     • This is a YouTube video collection for the Machine Learning course taught by Andrew Ng.
  4. Awesome Optimal Transport [link];
  5. Plenty of textbooks and videos available online.

Learning Outcomes

As a result of completing this course, the students will be able to:

• understand Neural ODE;
• understand generative models such as autoencoders, normalizing flow, Generative Adversarial networks (GAN);
• Apply the generative models to some mathematical problems.

AI and ChatGPT (University policies)

ChatGPT is strongly recommended in the daily study. But the use of AI and ChatGPT is completely forbidden for exams.

Average minimum amount of independent, out-of-class, learning expected per week

More than 2/3 of the time you devote to this class should take place outside the classroom (lecture and recitation). Even the best students in the class should plan on spending an average of at least 6 hours a week on homework and other studying. Students who struggle with the material may need to spend more time in order to earn a grade they will find acceptable.

Calendar

---

                                                                                    Wednesday

---

Week01. Aug26-Aug30                         Aug28: Intro to Pytorch, Numerical ODEs (code:[1],[2],[3])

---

Week02. Sep02-Sep06                        Sep04: Numerical ODEs

---

Week03. Sep09-Sep13                         Sep11: Neural ODEs: CNF (code:[4])

---

Week04. Sep16-Sep20                          Sep18: Neural ODEs

---

Week05. Sep23-Sep27                          Sep25: Intro to generative model

---

Week06. Sep30-Oct04                       Oct02: Convolutional Neural Network (CNN) [slides][code]

---

Week07. Oct07-Oct11                           Oct09 (midterm) and Autoencoders

---

Week08. Oct14-Oct18                            Oct16: Autoencoders [slides][code1][code2]

---

Week09. Oct21-Oct25                           Oct23: Generative Adversarial Netwrok (GAN) [slides][code]

---

Week10. Oct28-Nov01                             Oct30: Wasserstein GAN [code]

---

Week11. Nov04-Nov08                            Nov06: Normalizing Flow

---

Week12. Nov11-Nov15                               Nov13: Normalizing Flow

---

Week13. Nov18-Nov22       Nov20: TrajectoryNet

---

Week14. Nov25-Nov29                           Thanksgiving

---

Week15. Dec02-Dec06       Dec04 (final) and Optimal Transport

---

Week16. Dec09-Dec13      

---

NOTE: In accordance with university policy, the final exam will be given during the final exam period (Dec 12-17). There will be no accommodation for Christmas flight before scheduled final exam. 

Homework

• No homework assignments.

Exams

• If you have a legitimate conflict with the test dates and times (such as Student-athletes accommodation and Religious holidays), please contact the instructor as soon as possible, do not wait until shortly (within 24 hours) before the test.
• If you miss a test because of an illness, you must inform the instructor before the test, and get a note from your doctor in order to be allowed to make the test up at a later date. Unexplained missed tests will not be excused or allowed to be made up. 
• Assistance of any type (notes in any form, books, calculator, smartphone apps, etc.) is strictly banned during exams. Using the work of others on exams is strictly prohibited.
• One in-class midterm exam is scheduled at Oct 09.
• Final exam schedule will be announced later.

Grading

Your course grade will be determined by your cumulative average at the end of the term and will be based on the following scale:

	A	A-	B+	B	B-	C+	C	C-	D+	D	D-
Scale	95%	90%	87%	83%	80%	77%	73%	70%	67%	63%	60%

Your cumulative average will be the following weighted average:

	In-class performance	Midterm	Final
Scheme	30%	30%	40%

Academic Integrity Code

Academic integrity is an essential part of the educational process, and all members of the GW community take these matters very seriously. As the instructor of record for this course, my role is to provide clear expectations and uphold them in all assessments. Violations of academic integrity occur when students fail to cite research sources properly, engage in unauthorized collaboration, falsify data, and otherwise violate the Code of Academic Integrity. If you have any questions about whether particular academic practices or resources are permitted, you should ask me for clarification. If you are reported for an academic integrity violation, you should contact Conflict Education and Student Accountability (CESA), formerly known as Student Rights and Responsibilities (SRR), to learn more about your rights and options in the process. Consequences can range from failure of assignment to expulsion from the University and may include a transcript notation. For more information, refer to the CESA website at students.gwu.edu/code-academic-integrity or contact CESA by email cesa@gwu.edu or phone 202-994-6757.

University policy on observance of religious holidays

Students must notify faculty during the first week of the semester in which they are enrolled in the course, or as early as possible, but no later than three weeks prior to the absence, of their intention to be absent from class on their day(s) of religious observance. If the holiday falls within the first three weeks of class, the student must inform faculty in the first week of the semester. For details and policy, see provost.gwu.edu/policies-procedures-and-guidelines.

Use of Electronic Course Materials and Class Recordings

Students are encouraged to use electronic course materials, including recorded class sessions, for private personal use in connection with their academic program of study. Electronic course materials and recorded class sessions should not be shared or used for non-course related purposes unless express permission has been granted by the instructor. Students who impermissibly share any electronic course materials are subject to discipline under the Student Code of Conduct. Contact the instructor if you have questions regarding what constitutes permissible or impermissible use of electronic course materials and/or recorded class sessions. Contact Disability Support Services at disabilitysupport.gwu.edu if you have questions or need assistance in accessing electronic course materials.

Academic Commons

Academic Commons is the central location for academic support resources for GW students. To schedule a peer tutoring session for a variety of courses visit go.gwu.edu/tutoring. Visit academiccommons.gwu.edu for study skills tips, finding help with research, and connecting with other campus resources. For questions email academiccommons@gwu.edu.

GW Writing Center

GW Writing Center cultivates confident writers in the University community by facilitating collaborative, critical, and inclusive conversations at all stages of the writing process. Working alongside peer mentors, writers develop strategies to write independently in academic and public settings. Appointments can be booked online at gwu.mywconline.

Support for students in and outside the classroom

• Disability Support Services (DSS) 202-994-8250: Any student who may need an accommodation based on the potential impact of a disability should contact Disability Support Services at disabilitysupport.gwu.edu to establish eligibility and to coordinate reasonable accommodations.
• Student Health Center 202-994-5300, 24/7: The Student Health Center (SHC) offers medical, counseling/psychological, and psychiatric services to GW students. More information about the SHC is available at healthcenter.gwu.edu. Students experiencing a medical or mental health emergency on campus should contact GW Emergency Services at 202-994-6111, or off campus at 911.

GW Campus Emergency Information

GW Emergency Services: 202-994-6111

For situation-specific instructions, refer to GW’s Emergency Procedures guide. 

GW Alert

GW Alert is an emergency notification system that sends alerts to the GW community. GW requests students, faculty, and staff maintain current contact information by logging on to alert.gwu.edu. Alerts are sent via email, text, social media, and other means, including the Guardian app. The Guardian app is a safety app that allows you to communicate quickly with GW Emergency Services, 911, and other resources.  Learn more at safety.gwu.edu.

Protective Actions

GW prescribes four protective actions that can be issued by university officials depending on the type of emergency. All GW community members are expected to follow directions according to the specified protective action.  The protective actions are Shelter, Evacuate, Secure, and Lockdown (details below).  Learn more at safety.gwu.edu/gw-standard-emergency-statuses.

Questions

Everyone is strongly encouraged to ask questions during class, and during office hours! Should you need further assistance, you may consider hiring a tutor (the department keeps a list of tutors).