SearchNEU: The Modern Alternative to Northeastern’s Course Catalog
by Pranav Phadke and Ryan Hughes
Navigating Northeastern University’s course catalog can be daunting, especially with the limitations of the Banner system. This article explores the challenges associated with Banner, how Ryan Hughes (our founder) tackled these issues with the creation of SearchNEU, and how this innovative app has evolved with the contributions of Northeastern students. Search NEU started in 2017, grew to 50k+ users by 2019, and has been a staple at NEU ever since.
Difficulties using Northeastern’s official Course Catalog (Banner)
Banner, the course catalog application developed by Ellucian, is notoriously cumbersome. Accessing it requires logging in through Northeastern’s student hub, and it lacks mobile responsiveness, limiting searches to a computer. The search process itself is tedious. For instance, to find “Fundamentals of Computer Science 1,” you need to know its course number (2500) and subject (Computer Science) to find it in Banner.
Banner’s user interface is antiquated and resembles a pre-2000s spreadsheet. Being an outsourced application, the Northeastern community cannot make feature requests to improve the user experience or address privacy concerns. Additionally, the site is prone to outages, leaving students without a reliable, Northeastern-provided alternative for course information.
Another major shortcoming with Banner is its inability to notify students when seats open up in fully booked classes.This makes course registration frustrating, as students must constantly check for openings, often missing out due to the unavailability of timely alerts.
SearchNEU: A Better Solution
SearchNEU, often referred to as “Search”, is a search engine designed for easier navigation of class and professor information. It is mobile-responsive, requires no login, and offers notifications when seats in previously full sections become available.
The origin story
The story begins in 2016 when Ryan Hughes and some friends wanted to register for specific classes for the next semester. However, by the time they could register, those classes were fully booked. When Ryan asked the registrar for advice, they recommended constantly checking NEU’s course registration website to see if a seat opened up. Instead of manually checking the site every day, Ryan developed a script to automatically check and send notifications if it found a seat. This initial script was a success and widely used by many Northeastern students.
The first version of Search NEU
Ryan then started addressing various usability issues, which required pulling significantly more data from Northeastern’s website. Since the website lacked an API, Ryan built web scrapers that created objects representing semesters, courses, and sections. The entire course catalog was scraped every 24 hours, and a smaller script checked for open seats every five minutes.
This initial version of the app was built on a single t3.medium on AWS. Cloudflare was used as a CDN, Travis CI was used for CI and notifications were sent to users with Facebook Messenger’s API. The data itself was stored in Elastic Lunr.js on the backend and user data was stored in Firebase. Google analytics and Amplitude were used for analytics.
This first version of SearchNEU was a success and quickly spread throughout Northeastern.
The initial version looked like this:
SearchNEU’s Evolution
A few years later, in 2019, Ryan was graduating college and was looking for a group of students to continue the project. Around the same time, a group of Northeastern students (Ryan Drew, Da-Jin Chu, Mitch Gamburg) founded a student-led software consultancy called Sandbox. Their goal was to develop industry standard software for Northeastern researchers, professors, and the overall student community.
Sandbox proved to be an excellent organization to pass SearchNEU to because the founding team shared the same goals and had the necessary expertise. They adopted SearchNEU as their cornerstone project and initiated several other projects.
The initial Sandbox team continued its development and maintenance after Ryan’s graduation. The team worked on the site for a few semesters, and then other teams in Sandbox picked up the project after. Over 36 individuals have contributed across the frontend and backend repositories of the application. As a result, Search has been well maintained and improved since the application’s initial launch.
Timeline of SearchNEU over the last 5 years
2019
- Transitioned infrastructure to Kubernetes and Docker on AWS
- Built a GraphQL API for GraduateNU to utilize course-catalog data
- Migrated from Firebase to PostgreSQL
- Migrated from ElasticLunr.js to ElasticSearch
- Migrated frontend to Vercel
2020
Updated scrapers for new Banner versions and added support for CPS and LAW catalogs
- New version of Banner is depreciated right as new semester classes are released. Team rushes to update the scrapers to match the new Banner version.
- Support for CPS and LAW was also added to SearchNEU. Previously, Search only supported the undergraduate course catalog. There are two other colleges at Northeastern, the College of Professional Studies (CPS) and the Law school (LAW).
- The design of Search NEU was re-done as well, screenshots below
2021
- Transitioned from Facebook Messenger to Twilio for notifications. Facebook switched to only offering the messenger service to verified businesses.
- Enhanced scraper support for additional colleges
2022
- Rewrote backend code in TypeScript
- Big focus on improving Mobile responsiveness
- Added phrase searching to elastic search querying
2023
- Replace deprecated requests library
- Implemented an API-error page
- Created a DumpProcessor. Previously, the updater only updates seat count and meeting information for class sections that were already scraped and are in the database. Sometimes new sections are added mid semester. Those should also become scrapped and added to the database. The creation of the DumpProcessor fixes this issue.
- Updated backend documentation
- Updated to Node.js version 18 as required by Vercel. Updated Next.js, React.js, and other dependencies as well.
2024
- Mark past terms as inactive so that the updater doesn’t run on them and notifications aren’t allowed for them either
- Updated the sign-in flow for notifications
- Created a subscription page to manage all section notifications
Testimonials
- “SearchNEU is better than Banner in basically every way. It loads faster, offers flexible search, and provides notifications for open seats. It also has a mobile-friendly layout, making it easy to check course information on the go.” — Northeastern Student
- “I’m a CS faculty and assistant director at NU Vancouver. SearchNEU is essential for my workflow. Banner’s UI is outdated, but SearchNEU is responsive and extremely useful. Kudos to everyone who maintains it.” — CS Faculty
- “As an associate dean, I frequently use SearchNEU for enrollment information. It’s easier to access than Banner, reliable, and presents information clearly.” — Associate Dean
- “SearchNEU’s convenience, speed, and better UI make it perfect. No need for Duo authentication to look for classes.” — Northeastern Student
- “I missed the summer class registration deadline, but SearchNEU’s SMS reminders helped me secure the classes I needed to graduate on time.” — Northeastern Student
Conclusion
SearchNEU has proven to be a game-changer for the Northeastern community, offering a modern, user-friendly alternative to the cumbersome Banner system. Through the dedication and innovation of students and teams like Sandbox, SearchNEU has continuously evolved to meet the needs of its users, providing reliable course information and timely notifications. Its journey from a simple script to a robust application with thousands of users underscores the impact of student-driven solutions in enhancing campus life. As SearchNEU continues to grow and adapt, it remains an essential tool for navigating course registration at Northeastern University.
