Firewalls, Security Groups and Denial of Service

Topics Covered

The Filtering Layers a Packet Crosses

The layers, in the order they apply

What each layer can see

Stateful and stateless, and the rules each needs

Default deny, and what to allow

Order of evaluation within a layer

Verifying the effective policy rather than the rules

Network policy inside a cluster

Where encryption termination sits

Egress paths and the filters on them

Protocol and flag matching

A packet reaching an application has passed several independent filters, each owned by different people and configured in different places. Knowing which layers exist, in what order they apply and what each can decide is the difference between a confident change and a guess.

The layers, in the order they apply

From the outside in, a request typically crosses a provider's edge protection, a load balancer with its own listener rules, a subnet-level access list, an instance-level security group, the host's own firewall, and finally the application.

Each is evaluated independently and any one of them dropping the packet ends the journey. There is no negotiation between them, so a permissive rule at one layer does not override a restrictive rule at another, and the effective policy is the intersection of all of them.

The practical consequence is that opening a port is rarely a single change. A new port needs permitting at every layer that filters it, and a change that appears to have no effect is usually a change at the wrong layer.

What each layer can see

A filter can only decide on what it can read, and that determines which layer a given rule belongs at.

A packet filter sees addresses, ports and protocol flags, so it can allow a network or block a port and nothing more. A load balancer terminating the connection sees the request, so it can route by path or host and reject by method. An application-layer filter sees the full request including headers and body, so it can decide on content, patterns and rates.

Pushing a decision to the layer that can see what it needs, and no further out, is the design principle. Blocking a specific request pattern at a packet filter is impossible; blocking an entire network at an application filter works but wastes the resources of everything in front of it.

The effective policy is the intersection of every layer, and each layer can only decide on what it can read, which is what determines where a rule belongs.

Stateful and stateless, and the rules each needs

This distinction decides how many rules a change requires, and misunderstanding it is the most common configuration error.

A stateful filter tracks connections, so permitting a connection permits its responses automatically. One inbound rule on a port is complete, and the return traffic from that port to an arbitrary high port on the client needs no mention at all.

A stateless filter evaluates each packet alone, so an inbound rule covers only the request direction. The responses, leaving the port toward an ephemeral port on the client, need their own outbound rule spanning the whole ephemeral range. Omitting it produces a connection that appears to establish and never completes, which presents as a timeout and gets diagnosed as a routing problem.

Default deny, and what to allow

Every filter has a default, and the only safe default is to deny.

With a default-deny posture, the rules are a complete list of what is permitted, so reading them tells you the exposure. With default allow, the rules are a list of exclusions and the exposure is everything not mentioned, which cannot be verified by reading anything.

The allowances worth being deliberate about are inbound rules restricted to the narrowest source that works, outbound rules restricted at all, which most deployments neglect, and administrative access restricted to a bastion or an identity-based mechanism rather than an address range. Outbound restriction is the one that matters for containment, because it is what stops a compromised instance from reaching anything it likes.

Order of evaluation within a layer

Within one filter, rule order sometimes matters and sometimes does not, and assuming the wrong model produces surprises.

Security groups are a set of allowances with no ordering: a packet is permitted if any rule allows it, and there are no deny rules. Subnet access lists are ordered and include denies, so they are evaluated in sequence until one matches, which means a broad allow placed before a specific deny makes the deny unreachable.

Host firewalls are ordered chains with a default policy at the end, and a rule inserted after a matching accept has no effect. So the question to settle for any filter you are editing is whether it is an unordered allowance set or an ordered sequence, because the same change produces different results.

Verifying the effective policy rather than the rules

Reading rules tells you what someone intended, and testing tells you what is true.

The tests worth having are a connection attempt from a permitted source that must succeed, and one from a source that must fail. The second is the one that gets skipped, and it is the one that catches a rule accidentally allowing the world, which is the most common serious misconfiguration there is.

Provider tooling that evaluates reachability between two points across every layer is considerably faster than reasoning about it, and flow logs answer the question after the fact by recording accepted and rejected flows per interface. Between them, there is little excuse for guessing whether a path is open.

Network policy inside a cluster

A container platform adds another filtering layer with its own model, and it is the one most often left wide open.

By default, pods in a cluster can reach every other pod, which means a compromise anywhere reaches everything. Network policy restricts that, selecting pods by label and permitting traffic from named sources, and it is enforced by the networking plugin rather than by the platform itself, so a cluster whose plugin does not implement it silently ignores every policy written.

The two practical points are to confirm the plugin enforces policy before relying on it, and to adopt a default-deny policy per namespace and then allow the paths that are needed. Writing allowances without the default deny achieves nothing, since everything was already permitted.

Where encryption termination sits

The point at which encryption ends decides what each later layer can filter on, so it is a filtering decision as much as a security one.

Terminating at the edge lets every layer behind it read requests, which enables routing, caching and content-based filtering, at the cost of the internal hops being unencrypted unless re-established. Passing encrypted traffic through to the application preserves confidentiality end to end and blinds every intermediate filter, which means an application-layer rule set cannot work.

Most deployments terminate at the balancer and re-encrypt to the backends, which gives both. Knowing which arrangement is in use answers why a content-based rule has no effect, which is otherwise a confusing afternoon.

Egress paths and the filters on them

Outbound traffic crosses its own set of layers, and they are considered less often than the inbound ones.

A private instance reaches the internet through a translation gateway, which has its own capacity and its own source port pool. Traffic to a provider's managed service may go through a gateway endpoint or an interface endpoint, each with its own policy attached. An egress proxy, where one exists, can filter by destination name rather than address, which is the only way to express an allowance for a service whose addresses change constantly.

The consequence for debugging is that an outbound failure has as many candidate layers as an inbound one, and the endpoint policies are the ones most often forgotten because they live somewhere nobody thinks to look.

Protocol and flag matching

A packet filter can match on more than addresses and ports, and the extra criteria occasionally matter.

Matching on the protocol separates connection-oriented from connectionless traffic, which is how a connectionless service is kept off a public interface. Matching on connection state, where the filter supports it, allows established and related traffic without a broad rule. Matching on flags is how an initial connection packet is distinguished from the rest, which is what makes a stateless approximation of statefulness possible.

These are worth knowing mostly so that an existing rule set is readable, since a rule permitting only established traffic reads as permissive and is not. Writing new rules this way is rarely necessary where a stateful filter is available.

Level Expectations

A senior answer treats the effective policy as the intersection of every layer and names the layers rather than saying firewall. It distinguishes stateful filters, where one inbound rule is complete, from stateless ones, where the response needs its own outbound rule across the ephemeral range. And it insists on testing both directions of the expectation: that a permitted source connects, and that a forbidden one does not, because the second test is the one that catches a rule accidentally open to the world.