Scholastic

Scholastic was founded in 1920 as a single classroom magazine. Scholastic books and educational materials are in tens of thousands of schools and tens of millions of homes worldwide, helping to Open a World of Possible for children across the globe. Today Scholastic is a publishing, education, and media company known for publishing, selling, and distributing books and educational materials for schools, teachers, parents, and children. Products are distributed to schools and districts, consumers through the schools via reading clubs and fairs, and through retail stores and online sales. The business has three segments: Children Book Publishing & Distribution (Trade, Book Clubs and Book Fairs), Education, and International.

Scholastic also publishes instructional reading and writing programs, and offers professional learning and consultancy services for school improvement. Fun fact – Clifford the Big Red Dog serves as the mascot for Scholastic.

Challenge

  • Slow time to market, any changes to business processes/reports on AS400 data warehouse (AS400) taking several months impeding agility
  • Unable to meet internal SLAs due to performance of daily ETL (5,500 AS/400, 350+ SQL Server) processes
  • Scalability limitations with SQL Server Analysis Services (SSAS) for dashboards/reports
  • Limited ability to perform self-service business intelligence
  • Hiring and retaining talent for managing legacy systems

Solution Delivered to Meet Challenges

  • Delivered multiple AWS analytics workloads to the business sponsors as a proof of concept (POC) to demonstrate that the business would get better information/ knowledge to run their business
  • Used Native AWS solutions with Redshift as the primary Data Warehouse, EMR/DataPipeline as ETL and Tableau as the consumption layer
  • Reduce 36-month project timeline to 18 months by parallelizing the work effort

Architecture Diagram

Scholastic – AWS Database Migration

AWS Services Used

  • AWS Redshift
  • AWS Redshift Spectrum
  • AWS S3
  • AWS EC2
  • AWS EMR
  • AWS IAM
  • AWS DynamoDB
  • AWS RDS
  • AWS Glue
  • AWS SNS
  • AWS Lambda
  • AWS ElastiCache
  • AWS VPC
  • AWS Data Pipeline
  • AWS CloudWatch
  • AWS Step Functions

Database Workload Migration Details

  • Approx. 5 TB of data migrated (Batch+Full-load+Realtime)
  • We have a total of 5 main sources. These include:
    • DB2: approximately 2000+ tables were migrated
    • SQLServer: approximately 60+ table were migrated
    • FTP: approximately 100+ tables were migrated
    • Other source (Salesforce, Adobe, Customer DBs data coming realtime from Kafka, Restful APIs): approximately 400+ table were migrated
    • SAS 360 Discover: approximately 20 tables were migrated
  • Target:
    • Redshift having 2500+ table

Migration Patterns Used

  • We prepare migration yaml to deploy build using CI/CD framework
  • Schedule a job by analysing the ETL pipeline flow
  • Prepare a backout plan if anything goes wrong

Migration Tools Used

  • Sqoop
  • AWS Data Pipeline

Third-Party Tools Used

  • Jenkins
  • Apache Sqoop
  • Kafka
  • Redis
  • AtScale

About NorthBay

We are a fast-growing, 100% AWS focused onshore/offshore AWS Premier Consulting Partner, supporting our customers to accelerate the reinvention of their applications and data for a Cloud-native world. Our >350 AWS Certified Employees excel in developing and deploying Database & Application Migrations, Data Lakes and Analytics, Machine Learning/AI, DevOps and Application and Data Modernization/Development that drive measurable business impact.