Big Data questions and answers

Spark, Hadoop and batch processing at volume. Page 9 of 9.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Answers 481-495

…

Course

Beginner

27 lessons

10 hours

System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course

Deep Learning

Neural network architectures, training dynamics and GPU execution.

TensorFlow & Keras

Graph construction, layers, training loops and TensorBoard.

PyTorch

Tensors, autograd and the PyTorch training loop.

Natural Language Processing

Tokenisation, embeddings, language models and text pipelines.

Computer Vision

Image pipelines, detection and segmentation.

Machine Learning

Model training, evaluation and the classical algorithm toolkit.

Data Science

Pandas, NumPy and the analysis and visualisation stack.

Kafka & Streaming

Topics, consumer groups, stream processing and message queues.

Kubernetes

Pods, services, Helm and cluster operations.

Docker & Containers

Images, volumes, Compose and container networking.

AWS & Cloud

EC2, S3, Lambda, DynamoDB and cloud infrastructure.

Distributed Systems

Replication, consistency, microservices and scale.

Databases

SQL and NoSQL modelling, indexing and query performance.

Big Data

Spark, Hadoop and batch processing at volume.

DevOps & Deployment

Pipelines, servers, observability and release mechanics.

Networking & APIs

HTTP, REST, sockets and service-to-service calls.

Security & Auth

Authentication, authorisation, tokens and transport security.

Algorithms

Sorting, searching, recursion, graphs and dynamic programming.

Data Structures

Arrays, maps, trees, heaps and how to pick between them.

Complexity & Performance

Big O, time and space trade-offs, and making code faster.

Math for Programmers

Number theory, combinatorics and computational geometry.

Object-Oriented Design

Classes, interfaces, inheritance and design patterns.

Concurrency

Threads, async/await, locks and parallel execution.

Python

Language semantics, the standard library and packaging.

Java

The JVM, collections, Spring and build tooling.

C# & .NET

C# language features, LINQ, ASP.NET and the runtime.

JavaScript & Web

The browser runtime, Node, and front-end frameworks.

iOS & Android

Swift, Kotlin, the mobile UI toolkits and app lifecycle.

C & C++

Pointers, memory, templates and the standard library.

Git & Version Control

Branching, rebasing, remotes and recovering from mistakes.

Testing

Unit tests, mocking and keeping a suite trustworthy.

Debugging & Errors

Reading stack traces and tracking down the actual cause.