Cloudwork / Mon, 29 Jun 2026 19:25:38 +0000 en-US hourly 1 https://wordpress.org/?v=7.1.2 /wp-content/uploads/2025/03/Frame-24-1-150x150.webp Cloudwork / 32 32 EC2 8th‑gen nested virtualization: VMs inside VMs without bare metal /ec2-8th-gen-nested-virtualization/ Mon, 29 Jun 2026 19:23:32 +0000 /?p=1520 AWS nested virtualization, finally on virtual EC2 For years, AWS customers who wanted to run a hypervisor inside an EC2 instance – to host VMs inside VMs were forced onto expensive bare‑metal instance types. With the 2026 release of nested virtualization on 8th‑gen Intel virtual EC2 instances, AWS now lets you run KVM or Hyper‑V inside standard C8i, M8i, R8i and related families without touching bare metal. In practice, this means you can build full lab environments, microVM platforms, and emulator farms using normal virtual instances that fit your price/performance envelope instead of specialized hardware hosts. What nested virtualization actually is Nested virtualization lets you run a hypervisor (for example KVM or Hyper‑V) inside an EC2 instance, and then create additional virtual machines inside that guest. AWS documentation describes a three‑layer architecture: the physical AWS infrastructure and Nitro hypervisor at level L0, your EC2 instance running its own hypervisor at L1, and one or more nested VMs at L2. The Nitro System passes processor‑level virtualization extensions like Intel VT‑x into the guest, so your L1 hypervisor can use hardware virtualization to run L2 VMs with reasonable performance. Supported EC2 instance families Nested virtualization is currently supported only on specific x86 instance types, primarily 8th‑generation Intel families. AWS lists support for C8i, M8i, R8i, their “flex” variants, and disk‑optimized siblings like C8id, M8id, R8id, as well as some 7th‑gen Intel types such as C7i, M7i, R7i, C7i‑flex, M7i‑flex, X8i, and I7i. Graviton (ARM) instances are not currently supported, so any nested virtualization design today has to target these Intel families. According to AWS, there is no additional EC2 surcharge for enabling nested virtualization itself, you pay the usual instance pricing. From bare metal only to virtual instances Before this update, the only officially supported way to run nested VMs on EC2 was to use bare‑metal instances, which expose the physical hardware directly and tend to be significantly more expensive. Many teams resorted to EC2 Mac instances and other low‑end bare‑metal types as a workaround just to get VT‑x access for test labs and microVM experimentation. With nested virtualization on virtual instances, AWS now exposes the same processor features via Nitro to standard EC2 types, so you can choose instance sizes based on workload requirements instead of being locked into bare metal purely for hardware virtualization. Supported hypervisors and tooling AWS explicitly supports KVM and Hyper‑V as the L1 hypervisors inside nested‑virtualization‑enabled EC2 instances. That covers the most common Linux and Windows workflows, where you might be running QEMU/KVM on Linux hosts or Hyper‑V on Windows Server or Windows workstations. In practice, this unlocks a wide toolchain: Docker Desktop, Windows Subsystem for Linux 2 (WSL2), Android Studio emulators, Firecracker, Kata Containers, and other microVM or VM‑based runtimes that rely on hardware virtualization extensions. Key use cases for DevOps and platform teams AWS and early adopters highlight several concrete use cases that nested virtualization on 8th‑gen EC2 enables. Enabling nested virtualization on EC2 AWS exposes nested virtualization as a CPU option on supported instance types. When launching a new instance via the API or CLI, you can set the NestedVirtualization=enabled flag in CpuOptions on compatible Intel families like c8i, m8i, or r8i. For existing instances, AWS documentation notes that you can turn on nested virtualization after stopping the instance, provided the instance type supports the feature. Once the instance is up, you install and configure KVM or Hyper‑V as usual, and then create L2 VMs using your preferred tooling (libvirt, virt‑manager, Hyper‑V Manager, etc.). Limitations and caveats you need to know There are a few important constraints and trade‑offs to factor into your designs. Architectural patterns unlocked by nested virtualization From an architectural point of view, nested virtualization on virtual EC2 closes a gap that has existed between AWS and other clouds that already supported VMs‑inside‑VMs on standard instance types. It means you can now keep complex virtualization‑dependent tooling – emulator farms, microVM platforms, and full lab environments – centralized on EC2 without bespoke bare‑metal clusters. You can design patterns like “hypervisor hosts as pets, nested VMs as cattle”: use C8i/M8i/R8i instances as L1 hypervisor nodes, spin up L2 VMs on demand for ephemeral dev/test environments, and tear them down as easily as containers. For organizations investing in multi‑tenant sandbox platforms or heavy QA pipelines, this drastically reduces the friction of moving those workloads into AWS. How Cloudwork can help you adopt this feature At Cloudwork, we already work with teams that want disposable lab environments, complex CI pipelines, and sandboxed domain‑intelligence workloads on AWS. Nested virtualization on 8th‑gen EC2 gives us a more flexible tool for those designs. We can help you: If you’ve been avoiding nested virtualization on AWS because bare‑metal instances were too costly, this 8th‑gen EC2 update is the moment to revisit that decision, and we can work with you to make sure your architecture, security model, and performance profile are ready for VMs inside VMs in the cloud.

The post EC2 8th‑gen nested virtualization: VMs inside VMs without bare metal appeared first on Cloudwork.

]]>
AWS nested virtualization, finally on virtual EC2

For years, AWS customers who wanted to run a hypervisor inside an EC2 instance – to host VMs inside VMs were forced onto expensive bare‑metal instance types. With the 2026 release of nested virtualization on 8th‑gen Intel virtual EC2 instances, AWS now lets you run KVM or Hyper‑V inside standard C8i, M8i, R8i and related families without touching bare metal.

In practice, this means you can build full lab environments, microVM platforms, and emulator farms using normal virtual instances that fit your price/performance envelope instead of specialized hardware hosts.

What nested virtualization actually is

Nested virtualization lets you run a hypervisor (for example KVM or Hyper‑V) inside an EC2 instance, and then create additional virtual machines inside that guest. AWS documentation describes a three‑layer architecture: the physical AWS infrastructure and Nitro hypervisor at level L0, your EC2 instance running its own hypervisor at L1, and one or more nested VMs at L2.

The Nitro System passes processor‑level virtualization extensions like Intel VT‑x into the guest, so your L1 hypervisor can use hardware virtualization to run L2 VMs with reasonable performance.

Supported EC2 instance families

Nested virtualization is currently supported only on specific x86 instance types, primarily 8th‑generation Intel families. AWS lists support for C8i, M8i, R8i, their “flex” variants, and disk‑optimized siblings like C8id, M8id, R8id, as well as some 7th‑gen Intel types such as C7i, M7i, R7i, C7i‑flex, M7i‑flex, X8i, and I7i.

Graviton (ARM) instances are not currently supported, so any nested virtualization design today has to target these Intel families. According to AWS, there is no additional EC2 surcharge for enabling nested virtualization itself, you pay the usual instance pricing.

From bare metal only to virtual instances

Before this update, the only officially supported way to run nested VMs on EC2 was to use bare‑metal instances, which expose the physical hardware directly and tend to be significantly more expensive. Many teams resorted to EC2 Mac instances and other low‑end bare‑metal types as a workaround just to get VT‑x access for test labs and microVM experimentation.

With nested virtualization on virtual instances, AWS now exposes the same processor features via Nitro to standard EC2 types, so you can choose instance sizes based on workload requirements instead of being locked into bare metal purely for hardware virtualization.

Supported hypervisors and tooling

AWS explicitly supports KVM and Hyper‑V as the L1 hypervisors inside nested‑virtualization‑enabled EC2 instances. That covers the most common Linux and Windows workflows, where you might be running QEMU/KVM on Linux hosts or Hyper‑V on Windows Server or Windows workstations.

In practice, this unlocks a wide toolchain: Docker Desktop, Windows Subsystem for Linux 2 (WSL2), Android Studio emulators, Firecracker, Kata Containers, and other microVM or VM‑based runtimes that rely on hardware virtualization extensions.

Key use cases for DevOps and platform teams

AWS and early adopters highlight several concrete use cases that nested virtualization on 8th‑gen EC2 enables.

  • Mobile app CI and emulator farms
    You can run Android emulators in CI/CD pipelines or large device farms inside nested VMs, without paying for bare‑metal instances or on‑prem hardware.
  • Automotive and hardware simulation
    In‑vehicle hardware simulations and automotive software validation can run in fully virtualized testbeds, with nested hypervisors hosting different firmware or OS images as separate VMs.
  • Developer workstations in the cloud
    Windows workstations with WSL2, Docker Desktop, and Hyper‑V based tooling can be moved into EC2 while retaining their virtualization‑dependent workflows.
  • MicroVM platforms and multi‑tenant lab environments
    Platforms based on Firecracker or similar microVM technologies can now run on standard EC2 virtual instances, instead of needing bare metal, which is a big deal for multi‑tenant lab setups and sandbox services.

Enabling nested virtualization on EC2

AWS exposes nested virtualization as a CPU option on supported instance types. When launching a new instance via the API or CLI, you can set the NestedVirtualization=enabled flag in CpuOptions on compatible Intel families like c8i, m8i, or r8i.

For existing instances, AWS documentation notes that you can turn on nested virtualization after stopping the instance, provided the instance type supports the feature.

Once the instance is up, you install and configure KVM or Hyper‑V as usual, and then create L2 VMs using your preferred tooling (libvirt, virt‑manager, Hyper‑V Manager, etc.).

Limitations and caveats you need to know

There are a few important constraints and trade‑offs to factor into your designs.

  • Performance and latency overhead
    AWS explicitly recommends that performance‑sensitive workloads that require hardware virtualization still consider bare‑metal instances, as nested virtualization adds extra overhead through multiple hypervisor layers.
  • Windows‑specific constraints
    On Windows instances, enabling nested virtualization automatically disables Virtual Secure Mode (Credential Guard), and hibernation is not supported. There is also a CPU‑count limit – nested virtualization is not supported on certain very large Windows configurations, such as some m8i.96xlarge shapes.
  • Security responsibilities
    AWS continues to handle “security of the cloud” (Nitro, host isolation, and physical infrastructure), but customers are responsible for “security in the cloud”. That includes the guest OS, the L1 hypervisor, L2 VMs, and everything running inside those virtual machines.
  • Instance type and region coverage
    At launch, nested virtualization is available in all commercial regions on the listed Intel instance families; other architectures like Graviton are not supported yet. You must choose a matching instance type and region or your NestedVirtualization flag will have no effect or won’t be present at all.

Architectural patterns unlocked by nested virtualization

From an architectural point of view, nested virtualization on virtual EC2 closes a gap that has existed between AWS and other clouds that already supported VMs‑inside‑VMs on standard instance types. It means you can now keep complex virtualization‑dependent tooling – emulator farms, microVM platforms, and full lab environments – centralized on EC2 without bespoke bare‑metal clusters.

You can design patterns like “hypervisor hosts as pets, nested VMs as cattle”: use C8i/M8i/R8i instances as L1 hypervisor nodes, spin up L2 VMs on demand for ephemeral dev/test environments, and tear them down as easily as containers. For organizations investing in multi‑tenant sandbox platforms or heavy QA pipelines, this drastically reduces the friction of moving those workloads into AWS.

How Cloudwork can help you adopt this feature

At Cloudwork, we already work with teams that want disposable lab environments, complex CI pipelines, and sandboxed domain‑intelligence workloads on AWS. Nested virtualization on 8th‑gen EC2 gives us a more flexible tool for those designs.

We can help you:

  • Identify which of your current bare‑metal or on‑prem hypervisor workloads can move to nested virtualization on C8i/M8i/R8i.
  • Design safe multi‑tenant architectures for microVM platforms and emulator clusters with appropriate isolation and observability.
  • Integrate nested virtualization into your Terraform and CI flows so that lab environments and test benches become fully reproducible.

If you’ve been avoiding nested virtualization on AWS because bare‑metal instances were too costly, this 8th‑gen EC2 update is the moment to revisit that decision, and we can work with you to make sure your architecture, security model, and performance profile are ready for VMs inside VMs in the cloud.

The post EC2 8th‑gen nested virtualization: VMs inside VMs without bare metal appeared first on Cloudwork.

]]>
S3 just became a filesystem: Amazon S3 Files for real-world workloads /amazon-s3-files-for-real-workloads/ Fri, 26 Jun 2026 12:52:01 +0000 /?p=1513 S3 just became a filesystem: Amazon S3 Files for real-world workloads Amazon S3 is no longer “just” object storage – with the new Amazon S3 Files feature, you can mount S3 buckets as real, shared file systems with POSIX semantics and NFS access. For DevOps and platform teams, that means many workloads that used to need EFS or FSx (plus a lot of data‑copy glue) can now run directly on top of S3. What is Amazon S3 Files? Amazon S3 Files is a feature that exposes any general‑purpose S3 bucket (or prefix) as a mountable file system that applications can access over NFS. Under the hood, it keeps S3 as the authoritative data store but presents files and directories with familiar POSIX semantics like hierarchical paths, permissions, and atomic renames. You can mount S3 Files from EC2, containers (ECS/EKS/Fargate), and even AWS Lambda functions, so file‑based and object‑based workloads can share one storage layer. Because the feature is embedded in S3 itself rather than being a client‑side hack, it is designed to support heavy concurrent access, high throughput, and real multi‑writer scenarios. How S3 Files works under the hood S3 Files connects your bucket to a file system implementation backed by high‑performance storage that sits between your compute and S3. Frequently accessed data and metadata are cached on this fast storage, while S3 continues to hold the durable, long‑term copy. A few important mechanics to understand:techforward+1 From an application’s perspective, you get a normal NFS file system with close‑to‑open consistency semantics, while S3 Files handles the heavy lifting of cache management, synchronization, and integration with S3. Performance characteristics that matter AWS’ published numbers highlight that cached workloads can hit very low latency and massive aggregate throughput. For hot data in the high‑performance layer, S3 Files targets sub‑millisecond to low single‑digit millisecond access times. On the throughput side, AWS documentation and early benchmarks talk about tens of thousands of concurrent connections per file system and up to several terabytes per second of aggregate read throughput across clients and buckets. A single client doing large sequential reads can achieve multi‑GiB/s by leveraging parallel S3 GETs when bypassing through to S3. In practice, this means workloads like media processing, ML training, and analytics that previously required carefully tuned EFS or FSx deployments can now run directly against S3 with very competitive performance, provided that access patterns are either cache‑friendly or sequential. Pricing and cost model for S3 Files S3 Files uses a pay‑for‑what‑you‑use model tied to your active working set rather than the total size of your bucket. You pay for: For large, direct‑to‑S3 reads (when S3 Files decides to serve data straight from S3), you pay normal S3 request and data transfer charges without additional file‑system usage fees for those reads. AWS positions this as a way to avoid the cost of running separate file systems plus sync pipelines, claiming that you can cut total storage and data‑movement cost by up to around 90% compared with environments that duplicate datasets across S3 and file services. Because there is no provisioned capacity or minimum commitment for the file‑system layer itself, the economics scale naturally with how much data is actually “hot” in your workloads rather than how large your bucket has grown over time. S3 Files vs Mountpoint, s3fs, EFS, and FSx If you’ve been running S3‑centric architectures for a while, you likely already have tools like Mountpoint for Amazon S3, s3fs‑fuse, or native S3 SDK usage in the mix. S3 Files sits in a different category. The key takeaway is that S3 Files lets you stop maintaining separate S3 and file‑system copies for the same data in the common case, while you still use EFS or FSx where protocol or performance characteristics truly demand them. Key use cases for S3 Files AWS and early adopters have already highlighted several classes of workloads where S3 Files shines. Some of the most compelling scenarios: For Cloudwork clients, we see strong alignment with data‑heavy SaaS, analytics platforms, and domain‑intelligence pipelines that already depend on S3 as their primary data lake but still juggle extra file systems for specific tools or teams. Architectural implications for platform and DevOps teams The biggest shift with S3 Files is architectural: you can now design around a single authoritative storage layer in S3 without splitting object and file worlds. Object‑oriented services, query engines, and lakehouse systems still talk to S3 using APIs, while file‑based applications and agents mount the same underlying data as a file system. This simplifies governance, because you manage one set of IAM policies, encryption keys, and lifecycle rules for both object and file access rather than keeping S3, EFS, and FSx copies in sync. It also cuts down on bespoke sync pipelines and one‑off connectors, which reduces failure modes and operational overhead for platform teams.l From a DevOps perspective, S3 Files is also a powerful migration and modernization tool: you can containerize workloads that previously relied on local or NFS storage, mount S3 Files inside those containers, and gradually refactor them to native S3 APIs over time – or keep them file‑based if that is sufficient. Getting started with S3 Files (high-level) The basic getting‑started workflow is intentionally straightforward. For production workloads, you’ll also want to tune file‑size thresholds, retention windows, and access patterns based on how your applications read and write data. When S3 Files is not the right fit As powerful as S3 Files is, it is not the universal answer to all storage problems, and AWS is clear that it complements rather than replaces other services. For everything else that already stores primary data in S3 or will soon move there, S3 Files becomes a very attractive default for file‑semantics access, especially when data duplication has been a persistent pain point. How Cloudwork can help you adopt S3 Files At Cloudwork, we work with teams that are already deep into S3‑centric architectures – from SaaS platforms and content sites to analytics and domain‑intelligence pipelines. S3 Files gives us a new, cleaner building

The post S3 just became a filesystem: Amazon S3 Files for real-world workloads appeared first on Cloudwork.

]]>
S3 just became a filesystem: Amazon S3 Files for real-world workloads

Amazon S3 is no longer “just” object storage – with the new Amazon S3 Files feature, you can mount S3 buckets as real, shared file systems with POSIX semantics and NFS access. For DevOps and platform teams, that means many workloads that used to need EFS or FSx (plus a lot of data‑copy glue) can now run directly on top of S3.

What is Amazon S3 Files?

Amazon S3 Files is a feature that exposes any general‑purpose S3 bucket (or prefix) as a mountable file system that applications can access over NFS. Under the hood, it keeps S3 as the authoritative data store but presents files and directories with familiar POSIX semantics like hierarchical paths, permissions, and atomic renames.

You can mount S3 Files from EC2, containers (ECS/EKS/Fargate), and even AWS Lambda functions, so file‑based and object‑based workloads can share one storage layer. Because the feature is embedded in S3 itself rather than being a client‑side hack, it is designed to support heavy concurrent access, high throughput, and real multi‑writer scenarios.

How S3 Files works under the hood

S3 Files connects your bucket to a file system implementation backed by high‑performance storage that sits between your compute and S3. Frequently accessed data and metadata are cached on this fast storage, while S3 continues to hold the durable, long‑term copy.

A few important mechanics to understand:techforward+1

  • Intelligent caching: When files are actively used, their contents and metadata are pulled into the high‑performance layer and served with low‑millisecond or sub‑millisecond latency.
  • Size‑aware hydration: There is a configurable file‑size threshold (AWS’s examples use a default around the hundreds of kilobytes) to decide when to fully hydrate vs lazily stream content.
  • Direct S3 bypass for large reads: For big, sequential reads (for example, multi‑MiB streaming workloads), S3 Files can serve data directly from S3 at maximum throughput, only transferring the requested ranges.
  • Write‑back then sync: Writes land first on the high‑performance storage layer and are then batch‑synced back into S3, typically within seconds to a few minutes, with S3 remaining the source of truth.
  • Eviction of cold data: Data that has not been accessed for a configurable retention window is evicted from the high‑performance layer while remaining safely stored in S3.

From an application’s perspective, you get a normal NFS file system with close‑to‑open consistency semantics, while S3 Files handles the heavy lifting of cache management, synchronization, and integration with S3.

Performance characteristics that matter

AWS’ published numbers highlight that cached workloads can hit very low latency and massive aggregate throughput. For hot data in the high‑performance layer, S3 Files targets sub‑millisecond to low single‑digit millisecond access times.

On the throughput side, AWS documentation and early benchmarks talk about tens of thousands of concurrent connections per file system and up to several terabytes per second of aggregate read throughput across clients and buckets. A single client doing large sequential reads can achieve multi‑GiB/s by leveraging parallel S3 GETs when bypassing through to S3.

In practice, this means workloads like media processing, ML training, and analytics that previously required carefully tuned EFS or FSx deployments can now run directly against S3 with very competitive performance, provided that access patterns are either cache‑friendly or sequential.

Pricing and cost model for S3 Files

S3 Files uses a pay‑for‑what‑you‑use model tied to your active working set rather than the total size of your bucket. You pay for:

  • The capacity of the high‑performance storage layer that actually holds hot data.
  • File system access operations during import/export phases and metadata operations.

For large, direct‑to‑S3 reads (when S3 Files decides to serve data straight from S3), you pay normal S3 request and data transfer charges without additional file‑system usage fees for those reads. AWS positions this as a way to avoid the cost of running separate file systems plus sync pipelines, claiming that you can cut total storage and data‑movement cost by up to around 90% compared with environments that duplicate datasets across S3 and file services.

Because there is no provisioned capacity or minimum commitment for the file‑system layer itself, the economics scale naturally with how much data is actually “hot” in your workloads rather than how large your bucket has grown over time.

S3 Files vs Mountpoint, s3fs, EFS, and FSx

If you’ve been running S3‑centric architectures for a while, you likely already have tools like Mountpoint for Amazon S3, s3fs‑fuse, or native S3 SDK usage in the mix. S3 Files sits in a different category.

  • Versus Mountpoint for S3 / s3fs: Mountpoint and s3fs are client‑side solutions that translate file operations into S3 API calls, with various semantic gaps and performance trade‑offs (for example, limited support for certain file operations, weaker consistency guarantees, or poor behavior under heavy parallel writes). S3 Files, by contrast, is a managed service embedded into S3 with a dedicated cache and NFS server layer designed from the ground up to support full POSIX semantics for concurrent workloads.
  • Versus Amazon EFS: EFS is still a great fit when you want a classic regional NFS file system where the file system is the system of record and S3 may not be involved at all. S3 Files flips this around: S3 is the system of record, and the file system is a projection on top, ideal when your primary data lake or content repository already lives in S3.
  • Versus FSx (Lustre/Windows): FSx for Lustre and FSx for Windows are tuned for HPC and Windows file‑sharing scenarios, respectively, with their own protocols and integration patterns. S3 Files does not aim to replace those specialized services; instead, it targets the many workloads that already write to S3 but still need file access semantics for tools, agents, and legacy applications.

The key takeaway is that S3 Files lets you stop maintaining separate S3 and file‑system copies for the same data in the common case, while you still use EFS or FSx where protocol or performance characteristics truly demand them.

Key use cases for S3 Files

AWS and early adopters have already highlighted several classes of workloads where S3 Files shines. Some of the most compelling scenarios:

  • Machine learning training and inference
    Point training clusters directly at S3‑resident datasets and checkpoints without copying data to block volumes or EFS, and let multiple GPU nodes share the same mounted file system. Checkpoints and model artifacts end up in S3 automatically for audit and reuse.
  • Agentic AI and multi‑agent systems
    Multi‑agent workflows that expect a shared “scratch” filesystem can now use S3 Files as their working directory, while the same data lake remains accessible through object APIs, SQL engines, or vector search tools. This dramatically simplifies the storage story for AI pipelines that mix Python scripts, CLI tools, and managed services.
  • Artifact storage, CI/CD, and build systems
    Build pipelines and artifact registries can treat S3 buckets as shared file trees while still benefiting from S3’s durability, lifecycle policies, and cross‑region replication. This reduces complexity in systems that historically had to sync artifacts between S3 and a separate file store.
  • Legacy file‑based applications
    Existing applications that assume a POSIX file system – think older media processing stacks, ETL scripts, or configuration management systems – can be lifted into containers or serverless environments and pointed at S3 Files without rewriting their storage layer.

For Cloudwork clients, we see strong alignment with data‑heavy SaaS, analytics platforms, and domain‑intelligence pipelines that already depend on S3 as their primary data lake but still juggle extra file systems for specific tools or teams.

Architectural implications for platform and DevOps teams

The biggest shift with S3 Files is architectural: you can now design around a single authoritative storage layer in S3 without splitting object and file worlds. Object‑oriented services, query engines, and lakehouse systems still talk to S3 using APIs, while file‑based applications and agents mount the same underlying data as a file system.

This simplifies governance, because you manage one set of IAM policies, encryption keys, and lifecycle rules for both object and file access rather than keeping S3, EFS, and FSx copies in sync. It also cuts down on bespoke sync pipelines and one‑off connectors, which reduces failure modes and operational overhead for platform teams.l

From a DevOps perspective, S3 Files is also a powerful migration and modernization tool: you can containerize workloads that previously relied on local or NFS storage, mount S3 Files inside those containers, and gradually refactor them to native S3 APIs over time – or keep them file‑based if that is sufficient.

Getting started with S3 Files (high-level)

The basic getting‑started workflow is intentionally straightforward.

  1. Create an S3 Files file system for your bucket or prefix
    In the S3 console, you choose a bucket (or sub‑prefix) and create an associated file system, which links it to the high‑performance storage layer.
  2. Configure networking and mount targets
    AWS provisions mount targets in your VPC, and you ensure that your EC2 instances, containers, or on‑premises connections can reach these targets via appropriate subnets and security groups
  3. Mount from your compute environment
    On a Linux‑based instance or container, you install the appropriate utilities and then mount the file system using the S3 Files NFS‑compatible mount type. For example (AWS’ documentation uses similar commands): mkdir -p /mnt/s3files sudo mount -t s3files fs-XXXXXXXXXXXX:/ /mnt/s3filesAfter that, you can list, read, and write files under /mnt/s3files with standard tools, and the changes propagate back to S3
  4. Integrate with your IaC
    Over time, you’ll want to model S3 Files in Terraform/CloudFormation and codify IAM policies, mount targets, and security group rules, just as you do today for EFS or FSx. As the ecosystem matures, expect provider support and modules that make this repeatable across environments.

For production workloads, you’ll also want to tune file‑size thresholds, retention windows, and access patterns based on how your applications read and write data.

When S3 Files is not the right fit

As powerful as S3 Files is, it is not the universal answer to all storage problems, and AWS is clear that it complements rather than replaces other services.

  • HPC with specialized semantics: For ultra‑low‑latency HPC workloads that rely on Lustre semantics, FSx for Lustre remains the recommended solution.
  • Windows‑native workloads: Windows file shares and Active Directory‑integrated scenarios still point towards FSx for Windows File Server.
  • Block‑level requirements: Databases or workloads that need block‑device semantics and consistent single‑digit millisecond latency will continue to use EBS.
  • Offline or edge scenarios: If your workloads must operate with intermittent connectivity or fully on‑prem without direct S3 access, you’ll still need local storage or edge/appliance solutions.

For everything else that already stores primary data in S3 or will soon move there, S3 Files becomes a very attractive default for file‑semantics access, especially when data duplication has been a persistent pain point.

How Cloudwork can help you adopt S3 Files

At Cloudwork, we work with teams that are already deep into S3‑centric architectures – from SaaS platforms and content sites to analytics and domain‑intelligence pipelines. S3 Files gives us a new, cleaner building block for these systems.

We can help you:

  • Evaluate whether S3 Files is a good fit for your current mix of S3, EFS, FSx, and on‑premises file storage.
  • Design migration paths that eliminate redundant file systems and sync jobs without disrupting your existing workloads.
  • Update your infrastructure‑as‑code to provision and manage S3 Files alongside your buckets, VPCs, and compute.
  • Modernize legacy file‑based applications by lifting them into containers or serverless environments that mount S3 Files instead of local or NFS storage.

If you’re looking at your current S3 plus file‑storage setup and seeing a tangle of sync jobs, duplicated data, and special‑case exceptions, S3 Files is the moment to rethink that architecture – and we’d be happy to partner with you on that journey.

The post S3 just became a filesystem: Amazon S3 Files for real-world workloads appeared first on Cloudwork.

]]>
Server Security for Beginners: 10 Essential Steps to Protect Your First Server /server-security-for-beginners-10-essential-steps-to-protect-your-first-server/ Thu, 30 Apr 2026 11:39:20 +0000 /?p=1495 Server Security for Beginners: 10 Essential Steps to Protect Your First Server Renting your first server is exciting, until you realize that the moment it’s online, it is being scanned and probed by bots and opportunistic attackers.The good news is that a handful of practical steps will take you from “default and exposed” to “reasonably hardened” in a timely manner. In this guide, we’ll walk through a beginner‑friendly, real‑world checklist you can apply to any Linux server, whether you are on Hetzner, AWS, or another cloud provider.We’ll cover both operating‑system hardening and cloud‑level security controls like security groups, provider firewalls, VPNs, and Cloudflare Zero Trust. 1. Update Your Server Before Doing Anything Else Newly provisioned images are often weeks or months behind on patches, even if they were just created from your provider’s panel.Unpatched services are one of the most common ways attackers gain remote code execution within hours of a new vulnerability (CVE) being published. On day one: Security guidance consistently lists “apply OS and software updates immediately” as step one when bringing up a new server, for good reason. 2. Lock Down SSH: Keys, No Root, Limited Acces For most self‑hosted setups, SSH is the main way you manage your server and a prime target for brute‑force attacks and credential stuffing.Hardening SSH closes one of the easiest doors into your system. Implement the following: Industry best practices and security checklists repeatedly highlight SSH key‑based auth plus disabled root login as a baseline requirement for secure servers. 3. Turn On a Host‑Based Firewall (UFW, firewalld, nftables) A firewall should be configured to allow only the ports and protocols your application actually needs, blocking everything else by default.On Linux, that usually means enabling UFW (on Ubuntu/Debian) or firewalld/nftables (on many other distributions). At minimum: Guides from hosting providers and security vendors consistently treat “configure a local firewall” as a must‑do step when setting up a new server. 4. Use Cloud‑Level Firewalls and Security Groups Relying only on iptables/UFW on the instance itself is not enough in modern cloud environments.Cloud‑level firewalls add an extra protection layer that filters traffic before it ever reaches your VM’s network stack. Examples: Provider‑level firewalls and security groups are repeatedly mentioned in cloud security discussions as core building blocks for enforcing least‑privilege network access. 5. Add Brute‑Force Protection With Fail2ban or CrowdSec Even with keys, your server will see constant login attempts and vulnerability scans, especially on SSH and common web ports.Tools like Fail2ban or CrowdSec monitor logs for suspicious behavior and automatically block abusive IPs using firewall rules. Typical protections: Community and provider guides almost always pair local firewalls with Fail2ban‑style tools to reduce attack noise and slow down opportunistic attackers. 6. Secure Remote Access With VPNs and Cloudflare Zero Trust Where possible, keep your admin interfaces (SSH, control panels, internal tools) off the public internet entirely.Instead, expose them only through a VPN or a Zero Trust access layer. Two practical options: Zero Trust and VPN‑based controls are widely recommended as more modern alternatives to exposing admin surfaces directly with simple IP allow-lists. 7. Apply the Principle of Least Privilege to Users and Services Most production incidents are either configuration mistakes or misuse of powerful credentials, not just exotic exploits.Applying least privilege reduces the blast radius when something inevitably goes wrong. On a single server: In the cloud control plane: Security references consistently emphasise least privilege and regular access reviews as critical “day one” practices for securing new servers. 8. Encrypt Data in Transit and at Rest Even a well‑hardened server leaks value if traffic or sensitive data is exposed in plain text.Strong encryption—in transit and at rest—is a non‑negotiable baseline today. Key actions: Best‑practice server guides repeatedly call out TLS and encryption at rest as essential when storing customer data or handling payments, logins, or internal secrets. 9. Monitor Logs, Deploy IDS/IPS, and Set Up Alerts If you never look at logs, you’ll likely only discover a compromise once it impacts uptime or customers.Basic monitoring plus intrusion detection dramatically improves your chances of catching issues early. Start with: 10. Do Not Forget Backups and Recovery Testing Security is not just about keeping attackers out, it is also about being able to recover quickly when something breaks or data is lost.Ransomware, accidental rm -rf, application data corruption or a compromised admin account can all be mitigated by robust, tested backups. Follow these principles: Security and hosting guides consistently treat backups as part of the security story, not a separate operational concern. If you are just starting with your first server, focus on getting the basics right instead of chasing every advanced hardening trick.A server that is updated, locked down via SSH, protected with both host and cloud firewalls, behind a VPN or Zero Trust layer, monitored, and backed up is already far ahead of most internet‑connected machines. From there, you can iterate: add container isolation, mandatory access control (SELinux/AppArmor), secret management (Vault, AWS Secrets Manager), and more as your infrastructure grows.The important part is to treat server security as an ongoing practice, not a one‑time checklist you complete and forget. Cloudwork is here to help you design, automate, and harden your infrastructure from day one.

The post Server Security for Beginners: 10 Essential Steps to Protect Your First Server appeared first on Cloudwork.

]]>
Home

Server Security for Beginners: 10 Essential Steps to Protect Your First Server

Renting your first server is exciting, until you realize that the moment it’s online, it is being scanned and probed by bots and opportunistic attackers.
The good news is that a handful of practical steps will take you from “default and exposed” to “reasonably hardened” in a timely manner.

In this guide, we’ll walk through a beginner‑friendly, real‑world checklist you can apply to any Linux server, whether you are on Hetzner, AWS, or another cloud provider.
We’ll cover both operating‑system hardening and cloud‑level security controls like security groups, provider firewalls, VPNs, and Cloudflare Zero Trust.

1. Update Your Server Before Doing Anything Else

Newly provisioned images are often weeks or months behind on patches, even if they were just created from your provider’s panel.
Unpatched services are one of the most common ways attackers gain remote code execution within hours of a new vulnerability (CVE) being published.

On day one:

  • Update the package index and upgrade all packages. Note that every Linux distribution has its own package manager, so you should check the correct commands for your specific operating system (Ubuntu, Fedora, Amazon-Linux, etc.).
  • Enable automatic security updates where it makes sense (especially on single‑purpose servers).
  • Start with minimal server setup where possible. Remove software you don’t need to shrink the attack surface.

Security guidance consistently lists “apply OS and software updates immediately” as step one when bringing up a new server, for good reason.

2. Lock Down SSH: Keys, No Root, Limited Acces

For most self‑hosted setups, SSH is the main way you manage your server and a prime target for brute‑force attacks and credential stuffing.
Hardening SSH closes one of the easiest doors into your system.

Implement the following:

  • Use SSH keys instead of passwords: Generate a key pair locally, add the public key to the server, and disable password authentication in sshd_config.
  • Disable direct root login: Force admins to log in as a non‑root user and escalate via sudo when needed.
  • Restrict SSH by IP where possible: Limit SSH access to your static IP or VPN ranges, especially on production servers.
  • Optional: move SSH to a non‑default port: This does not replace real security controls, but it reduces noise from mass‑scanning bots. It’s a common practice for when you cant limit SSH access from static IPs or VPN ranges.

Industry best practices and security checklists repeatedly highlight SSH key‑based auth plus disabled root login as a baseline requirement for secure servers.

3. Turn On a Host‑Based Firewall (UFW, firewalld, nftables)

A firewall should be configured to allow only the ports and protocols your application actually needs, blocking everything else by default.
On Linux, that usually means enabling UFW (on Ubuntu/Debian) or firewalld/nftables (on many other distributions).

At minimum:

  • Default policy: deny incoming, allow outgoing.
  • Allow SSH from trusted sources.
  • Allow HTTP/HTTPS (80/443) if you are running a web app or API.
  • Close or explicitly block all other ports.

Guides from hosting providers and security vendors consistently treat “configure a local firewall” as a must‑do step when setting up a new server.

4. Use Cloud‑Level Firewalls and Security Groups

Relying only on iptables/UFW on the instance itself is not enough in modern cloud environments.
Cloud‑level firewalls add an extra protection layer that filters traffic before it ever reaches your VM’s network stack.

Examples:

  • AWS Security Groups:
    • Treat them as stateful, virtual firewalls attached to EC2 instances.
    • Allow only the ports your application needs (e.g. 80/443) from the internet (0.0.0.0/0), and restrict SSH to your IP or VPN CIDR.
  • Hetzner Cloud Firewall:
    • Define inbound rules for HTTP/HTTPS, and optionally SSH from trusted IPs.
    • Deny all other inbound traffic by default.
    • Reuse firewall profiles across multiple servers for consistent hardening.

Provider‑level firewalls and security groups are repeatedly mentioned in cloud security discussions as core building blocks for enforcing least‑privilege network access.

5. Add Brute‑Force Protection With Fail2ban or CrowdSec

Even with keys, your server will see constant login attempts and vulnerability scans, especially on SSH and common web ports.
Tools like Fail2ban or CrowdSec monitor logs for suspicious behavior and automatically block abusive IPs using firewall rules.

Typical protections:

  • SSH brute‑force attempts.
  • Repeated invalid HTTP requests or excessive 4xx/5xx responses.
  • Login abuse against control panels or web apps.

Community and provider guides almost always pair local firewalls with Fail2ban‑style tools to reduce attack noise and slow down opportunistic attackers.

6. Secure Remote Access With VPNs and Cloudflare Zero Trust

Where possible, keep your admin interfaces (SSH, control panels, internal tools) off the public internet entirely.
Instead, expose them only through a VPN or a Zero Trust access layer.

Two practical options:

  • Provider or self‑hosted VPN
    • Deploy WireGuard/OpenVPN and only allow SSH and admin HTTP ports from VPN subnets.
    • Many cloud platforms provide managed VPN or private networking primitives you can leverage.
  • Cloudflare Zero Trust
    • Put admin panels or internal dashboards behind Cloudflare Access.
    • Enforce identity‑aware policies (SSO, MFA, device posture checks) before any connection reaches your origin.
    • Combine this with Cloudflare WAF to filter malicious HTTP traffic at the edge.

Zero Trust and VPN‑based controls are widely recommended as more modern alternatives to exposing admin surfaces directly with simple IP allow-lists.

7. Apply the Principle of Least Privilege to Users and Services

Most production incidents are either configuration mistakes or misuse of powerful credentials, not just exotic exploits.
Applying least privilege reduces the blast radius when something inevitably goes wrong.

On a single server:

  • Create separate users for different roles instead of sharing root or a generic admin account.
  • Use sudo and carefully scoped groups for administrative operations.
  • Regularly audit accounts and disable unused logins.

In the cloud control plane:

  • Use IAM roles and policies (AWS) or similar constructs to limit what each service and user can do.
  • Avoid giving full admin rights to CI systems, deployment agents, or third‑party tools.

Security references consistently emphasise least privilege and regular access reviews as critical “day one” practices for securing new servers.

8. Encrypt Data in Transit and at Rest

Even a well‑hardened server leaks value if traffic or sensitive data is exposed in plain text.
Strong encryption—in transit and at rest—is a non‑negotiable baseline today.

Key actions:

  • HTTPS everywhere
  • Database and disk encryption
    • Enable encryption at rest for managed databases (e.g. AWS RDS, many hosted MySQL/PostgreSQL services).
    • Consider full‑disk encryption for self‑managed servers that handle especially sensitive data.

Best‑practice server guides repeatedly call out TLS and encryption at rest as essential when storing customer data or handling payments, logins, or internal secrets.

9. Monitor Logs, Deploy IDS/IPS, and Set Up Alerts

If you never look at logs, you’ll likely only discover a compromise once it impacts uptime or customers.
Basic monitoring plus intrusion detection dramatically improves your chances of catching issues early.

Start with:

  • Centralized logging server and application logs.
  • Simple alerting rules for repeated failed logins or abnormal traffic patterns.
  • Host‑based IDS tools (e.g. Wazuh) that detect suspicious file changes or behaviours.

10. Do Not Forget Backups and Recovery Testing

Security is not just about keeping attackers out, it is also about being able to recover quickly when something breaks or data is lost.
Ransomware, accidental rm -rf, application data corruption or a compromised admin account can all be mitigated by robust, tested backups.

Follow these principles:

  • Use automated, regular backups stored off‑server (ideally on another provider on a different region).
  • Encrypt backup data, especially when it contains customer information or secrets.
  • Periodically perform restore tests/drills so you know you can rebuild a server from backups under time pressure.

Security and hosting guides consistently treat backups as part of the security story, not a separate operational concern.

If you are just starting with your first server, focus on getting the basics right instead of chasing every advanced hardening trick.
A server that is updated, locked down via SSH, protected with both host and cloud firewalls, behind a VPN or Zero Trust layer, monitored, and backed up is already far ahead of most internet‑connected machines.

From there, you can iterate: add container isolation, mandatory access control (SELinux/AppArmor), secret management (Vault, AWS Secrets Manager), and more as your infrastructure grows.
The important part is to treat server security as an ongoing practice, not a one‑time checklist you complete and forget.

Cloudwork is here to help you design, automate, and harden your infrastructure from day one.

The post Server Security for Beginners: 10 Essential Steps to Protect Your First Server appeared first on Cloudwork.

]]>
How Cloudflare Caching Cut TTFB on cloudwork.sh (and Why It Matters for E‑commerce) /cloudflare-wordpress-cache-rules-ttfb/ Fri, 20 Mar 2026 21:01:12 +0000 /?p=1450 How Cloudflare Caching Cut TTFB on cloudwork.sh (and Why It Matters for E‑commerce) Time to First Byte (TTFB) is a foundational performance metric that measures how quickly a web server starts responding to a request. It is especially critical for e‑commerce and lead‑generation sites where every extra millisecond increases bounce rates and reduces conversions. This report explains what TTFB is, why it matters for modern e‑commerce websites, and how Cloudflare caching was used on cloudwork.sh to dramatically reduce global TTFB for static pages without breaking WordPress admin, logged‑in sessions, or sensitive endpoints. What TTFB Actually Measures TTFB is the time from when a browser sends an HTTP request until it receives the first byte of the response from the server. It includes multiple phases: DNS resolution, TCP/TLS connection setup, and the time the origin server spends generating and starting to send the response. Because all other page‑load work (downloading HTML, CSS, JavaScript, images, executing scripts, and rendering) can only start after that first byte arrives, TTFB is a strong early indicator of overall responsiveness. High TTFB often indicates slow backend processing, network latency between user and origin, or both. Why TTFB Matters for E‑commerce For e‑commerce, slow initial response time directly hurts both user experience and business metrics. In short: if the server takes nearly a second before sending the first byte, users start abandoning the page before they ever see meaningful content, and as a result, checkout funnels suffer. Baseline TTFB on cloudwork.sh (Before Caching) Cloudwork.sh was tested from multiple global locations using a TTFB measurement tool before enabling Cloudflare edge caching for HTML pages. All responses were cache MISS, meaning requests went directly to the origin server each time. On this uncached configuration, TTFB ranged roughly from 200–950 ms depending on region. Nearby European locations like Brussels and Helsinki (the site is hosted in Greece) showed TTFB in the low‑200 ms range, while long‑haul regions such as Sydney and Tokyo approached or exceeded 900 ms due to network distance. Origin processing time is not measured on this test as NGINX page caching was already enabled on the origin server. Optimized TTFB with Cloudflare Edge Caching After configuring targeted Cloudflare cache rules, static pages on cloudwork.sh (homepage, blog posts, marketing pages) were served from Cloudflare’s edge rather than from the origin for most visitors. The same global TTFB test was run again. In the optimized run, all locations reported a cache HIT, confirming that responses were served directly from Cloudflare’s POPs (points-of-presence). TTFB dropped to roughly 50–140 ms worldwide, with most regions under 80 ms even when they are geographically far from the origin server. This translated into a roughly 4–7× reduction in TTFB compared to the uncached baseline, pushing the site firmly into the “fast” category for a global audience. Why Cloudflare Improves TTFB for Static Pages Cloudflare operates a large global anycast CDN network, terminating user connections at the nearest edge location and serving cached copies of content directly from that POP. When an HTML page is cached at the edge, the user no longer has to wait for a full round‑trip to the origin server plus backend processing time. Instead, only the local edge connection and TLS handshake affect TTFB. For static or mostly‑static pages (homepages, landing pages, articles, and marketing content), this can dramatically reduce TTFB while also offloading CPU and bandwidth from the origin server. The key is to cache aggressively where it is safe, and bypass cache wherever content is user‑specific or sensitive. Designing Safe Cache Rules for WordPress A naive approach on Cloudflare is to create a single “Cache Everything” rule for the entire domain, but this is dangerous on a dynamic platform like WordPress. When Cloudflare caches all HTML indiscriminately, it may cache: This leads to stale or even confusing content being shown to the wrong users, cached admin bars on the public site, and broken session‑based flows such as carts and checkouts. Practical Takeaways for E‑commerce and Content Sites For e‑commerce stores and content‑driven sites, the pattern demonstrated on cloudwork.sh offers a pragmatic way to lower TTFB globally without sacrificing correctness or security.

The post How Cloudflare Caching Cut TTFB on cloudwork.sh (and Why It Matters for E‑commerce) appeared first on Cloudwork.

]]>
How Cloudflare Caching Cut TTFB on cloudwork.sh (and Why It Matters for E‑commerce)

Time to First Byte (TTFB) is a foundational performance metric that measures how quickly a web server starts responding to a request. It is especially critical for e‑commerce and lead‑generation sites where every extra millisecond increases bounce rates and reduces conversions.

This report explains what TTFB is, why it matters for modern e‑commerce websites, and how Cloudflare caching was used on cloudwork.sh to dramatically reduce global TTFB for static pages without breaking WordPress admin, logged‑in sessions, or sensitive endpoints.

What TTFB Actually Measures

TTFB is the time from when a browser sends an HTTP request until it receives the first byte of the response from the server. It includes multiple phases: DNS resolution, TCP/TLS connection setup, and the time the origin server spends generating and starting to send the response.

Because all other page‑load work (downloading HTML, CSS, JavaScript, images, executing scripts, and rendering) can only start after that first byte arrives, TTFB is a strong early indicator of overall responsiveness. High TTFB often indicates slow backend processing, network latency between user and origin, or both.

Why TTFB Matters for E‑commerce

For e‑commerce, slow initial response time directly hurts both user experience and business metrics.

  • Google and performance tooling typically classify a TTFB under 800 ms as “good” with anything above 800–1800 ms in the “needs improvement” range and slower responses considered poor.
  • Industry research shows that a 1 second delay in overall load time can reduce conversion rates by more than 5 percent and significantly increase bounce rates.
  • Visitors are especially impatient on mobile connections, where network variability amplifies the impact of high TTFB.

In short: if the server takes nearly a second before sending the first byte, users start abandoning the page before they ever see meaningful content, and as a result, checkout funnels suffer.

Baseline TTFB on cloudwork.sh (Before Caching)

WordPress Cloudflare cache before: high TTFB of 1200ms on landing page load

Cloudwork.sh was tested from multiple global locations using a TTFB measurement tool before enabling Cloudflare edge caching for HTML pages. All responses were cache MISS, meaning requests went directly to the origin server each time.

On this uncached configuration, TTFB ranged roughly from 200–950 ms depending on region. Nearby European locations like Brussels and Helsinki (the site is hosted in Greece) showed TTFB in the low‑200 ms range, while long‑haul regions such as Sydney and Tokyo approached or exceeded 900 ms due to network distance. Origin processing time is not measured on this test as NGINX page caching was already enabled on the origin server.

Optimized TTFB with Cloudflare Edge Caching

WordPress Cloudflare cache after: TTFB reduced to 150ms with optimized cache rules

After configuring targeted Cloudflare cache rules, static pages on cloudwork.sh (homepage, blog posts, marketing pages) were served from Cloudflare’s edge rather than from the origin for most visitors. The same global TTFB test was run again.

In the optimized run, all locations reported a cache HIT, confirming that responses were served directly from Cloudflare’s POPs (points-of-presence). TTFB dropped to roughly 50–140 ms worldwide, with most regions under 80 ms even when they are geographically far from the origin server.

This translated into a roughly 4–7× reduction in TTFB compared to the uncached baseline, pushing the site firmly into the “fast” category for a global audience.

Why Cloudflare Improves TTFB for Static Pages

Cloudflare operates a large global anycast CDN network, terminating user connections at the nearest edge location and serving cached copies of content directly from that POP. When an HTML page is cached at the edge, the user no longer has to wait for a full round‑trip to the origin server plus backend processing time. Instead, only the local edge connection and TLS handshake affect TTFB.

For static or mostly‑static pages (homepages, landing pages, articles, and marketing content), this can dramatically reduce TTFB while also offloading CPU and bandwidth from the origin server. The key is to cache aggressively where it is safe, and bypass cache wherever content is user‑specific or sensitive.

Designing Safe Cache Rules for WordPress

A naive approach on Cloudflare is to create a single “Cache Everything” rule for the entire domain, but this is dangerous on a dynamic platform like WordPress. When Cloudflare caches all HTML indiscriminately, it may cache:

  • Admin pages (wp‑admin, login, dashboard)
  • Logged‑in views that include personalized content or admin bars
  • Cart, checkout, or account pages on e‑commerce stores
  • Preview URLs and other nonce‑protected endpoints

This leads to stale or even confusing content being shown to the wrong users, cached admin bars on the public site, and broken session‑based flows such as carts and checkouts.

Practical Takeaways for E‑commerce and Content Sites

For e‑commerce stores and content‑driven sites, the pattern demonstrated on cloudwork.sh offers a pragmatic way to lower TTFB globally without sacrificing correctness or security.

  • Aim for a TTFB well under 800 ms, ideally in the 100–300 ms range for global visitors, by pushing cacheable HTML to the CDN edge.
  • Use explicit bypass rules for admin, login, previews, carts, checkouts, AJAX endpoints, and any page that depends heavily on per‑user state.
  • Use cookie‑based bypass for logged‑in sessions so editors, admins, and returning commenters always see fresh content and correct session state.

The post How Cloudflare Caching Cut TTFB on cloudwork.sh (and Why It Matters for E‑commerce) appeared first on Cloudwork.

]]>
Rethinking WordPress Infrastructure: The DevOps Imperative /wordpress-devops/ Thu, 17 Jul 2025 11:18:56 +0000 /?p=1044 Rethinking WordPress Infrastructure: The DevOps Imperative Major Brands Using WordPress Major brands leverage WordPress for their content and commerce: for example, TechCrunch (≈338,000 visits per day), sections of The New York Times (≈2 million daily visits), and even parts of Microsoft’s web presence (~16 million daily visits) run on WordPress. WordPress is also hugely popular for e-commerce, about one-third of all online shops are built on the WooCommerce platform (a WordPress plugin). The DevOps Paradox in WordPress Yet, there’s a paradox: Despite WordPress’s dominant presence on the web, the platform’s ecosystem has historically lagged in adopting modern DevOps practices. In many WordPress communities and deployments, critical DevOps principles like automated testing, continuous integration, scalable infrastructure, and proactive monitoring are often downplayed or overlooked. This DevOps gap can leave high-traffic WordPress sites struggling with performance bottlenecks, high hosting costs, and operational risks that more modern development workflows could mitigate. In this article, we’ll explore why WordPress infrastructure needs DevOps support, how DevOps practices can revolutionize WordPress scalability and reliability, and how you can structure your own approach to bring WordPress into the modern era of automation and efficiency. The Hidden Problem: WordPress’s DevOps Deficit WordPress’s greatest strength , its accessibility and ease of use, can also be a weakness when it comes to engineering rigor. The low barrier to entry means anyone can spin up a WordPress site or develop a plugin, which has fostered a huge community but also a prevalence of informal or outdated development practices. Seasoned developers have observed a “lack of standards and best practices in the WordPress world”. It’s common to find poorly coded themes or plugins “slapped together” without much regard for code quality, security, or performance. This isn’t just anecdotal: in community discussions, experienced WordPress professionals lament how many WordPress projects treat critical matters like security and optimization as afterthoughts, for example, considering a caching plugin a complete performance strategy, or ignoring proper security hardening until a disaster strikes. One core issue is that traditional WordPress development workflows often lack the automation and collaboration that DevOps encourages. Many WordPress sites are still updated by manually uploading files via FTP or by clicking “Update” in the dashboard, rather than using version control and automated deploy pipelines. In fact, WordPress’s architecture itself hasn’t fully embraced modern DevOps tooling. An illuminating “open letter” from a developer pointed out that WordPress’s core team has resisted key DevOps-enabling features: for instance, WordPress still has no official support for using Composer (PHP’s package manager) to install or update core and plugins. (For comparison, other PHP-based systems like Drupal have tightly integrated Composer into their workflows to manage dependencies and configuration.) The same letter notes how WordPress’s command-line tools (wp-cli) can’t perform certain tasks (like installing a language pack) in a headless or CI environment without a full site running, making true continuous integration difficult.In short, WordPress’s out-of-the-box tooling assumes a manual, one-server setup, which conflicts with the declarative, automated approach that DevOps teams prefer. The cultural side of the WordPress community also hasn’t emphasized DevOps historically. Many WordPress developers come from content or design backgrounds, not traditional software engineering, and may not be familiar with practices like infrastructure as code, containerization, or continuous delivery, and this is normal. As a result, a lot of WordPress development happens in “silos”,  a developer might work in isolation on code, a sysadmin (if one exists) manages servers separately, and there’s minimal collaboration or automation bridging the two. Why This Matters: Risks of Sticking to Traditional Approaches Why do these DevOps shortcomings in the WordPress world matter so much? Simply put, a lack of modern infrastructure practices can hurt both the performance and the profitability of a WordPress-based business. Let’s break down some of the concrete impacts: In summary, sticking to the traditional, ad-hoc way of managing WordPress might seem easier initially, but it exacts a toll in scalability, reliability, and cost efficiency as a site grows. The bigger and more complex your WordPress site (especially for high-traffic blogs or busy WooCommerce stores), the more painful and risky it becomes to operate without DevOps best practices in place. Next, let’s look at how embracing DevOps can turn these challenges into opportunities. Bringing DevOps to WordPress: What Does it Entail? “DevOps” may sound like a buzzword, but at its core it’s about bridging the gap between development and operations through better process and automation. In a WordPress context, adopting DevOps means shifting away from one-off manual work towards streamlined, repeatable workflows for building, testing, and deploying your site. It’s both a cultural change (developers, IT admins, and content teams collaborating more closely) and a technical change (using tools to automate tasks and manage infrastructure). Here are some key DevOps practices and how they can be applied to WordPress: In essence, DevOps for WordPress means applying the same rigor and automation that big tech companies use for their software, but tailored to the WordPress ecosystem. This might involve some upfront effort, adopting new tools and workflows, but the payoff is enormous for sites that have a lot at stake. The next sections will delve into one of the most critical areas that DevOps can address for WordPress sites: scalability of the hosting infrastructure. Scaling WordPress: From Overprovisioned Servers to Cloud Native One of the clearest benefits of bringing DevOps practices to WordPress is unlocking true scalability, the ability for your site’s infrastructure to grow (or shrink) on demand to match your traffic. Traditionally, many WordPress sites started on a single LAMP server (Linux, Apache, MySQL, PHP) and stayed that way. But as we’ve discussed, a single server has finite limits and can become a single point of failure. Let’s look at how modern infrastructure (often cloud-based and automated) can transform WordPress performance for high traffic scenarios. Vertical vs. Horizontal Scaling: Scaling essentially comes in two flavors. Vertical scaling (or “scaling up”) means you upgrade your server  e.g. move from a 4 CPU / 8 GB RAM machine to an 16 CPU / 64 GB RAM

The post Rethinking WordPress Infrastructure: The DevOps Imperative appeared first on Cloudwork.

]]>
Rethinking WordPress Infrastructure: The DevOps Imperative

Major Brands Using WordPress

Major brands leverage WordPress for their content and commerce: for example, TechCrunch (≈338,000 visits per day), sections of The New York Times (≈2 million daily visits), and even parts of Microsoft’s web presence (~16 million daily visits) run on WordPress.

WordPress is also hugely popular for e-commerce, about one-third of all online shops are built on the WooCommerce platform (a WordPress plugin).

The DevOps Paradox in WordPress

Yet, there’s a paradox: Despite WordPress’s dominant presence on the web, the platform’s ecosystem has historically lagged in adopting modern DevOps practices. In many WordPress communities and deployments, critical DevOps principles like automated testing, continuous integration, scalable infrastructure, and proactive monitoring are often downplayed or overlooked.

This DevOps gap can leave high-traffic WordPress sites struggling with performance bottlenecks, high hosting costs, and operational risks that more modern development workflows could mitigate.

In this article, we’ll explore why WordPress infrastructure needs DevOps support, how DevOps practices can revolutionize WordPress scalability and reliability, and how you can structure your own approach to bring WordPress into the modern era of automation and efficiency.

The Hidden Problem: WordPress’s DevOps Deficit

WordPress’s greatest strength , its accessibility and ease of use, can also be a weakness when it comes to engineering rigor. The low barrier to entry means anyone can spin up a WordPress site or develop a plugin, which has fostered a huge community but also a prevalence of informal or outdated development practices. Seasoned developers have observed a “lack of standards and best practices in the WordPress world”.

It’s common to find poorly coded themes or plugins “slapped together” without much regard for code quality, security, or performance. This isn’t just anecdotal: in community discussions, experienced WordPress professionals lament how many WordPress projects treat critical matters like security and optimization as afterthoughts, for example, considering a caching plugin a complete performance strategy, or ignoring proper security hardening until a disaster strikes.

One core issue is that traditional WordPress development workflows often lack the automation and collaboration that DevOps encourages. Many WordPress sites are still updated by manually uploading files via FTP or by clicking “Update” in the dashboard, rather than using version control and automated deploy pipelines. In fact, WordPress’s architecture itself hasn’t fully embraced modern DevOps tooling.

An illuminating “open letter” from a developer pointed out that WordPress’s core team has resisted key DevOps-enabling features: for instance, WordPress still has no official support for using Composer (PHP’s package manager) to install or update core and plugins. (For comparison, other PHP-based systems like Drupal have tightly integrated Composer into their workflows to manage dependencies and configuration.)

The same letter notes how WordPress’s command-line tools (wp-cli) can’t perform certain tasks (like installing a language pack) in a headless or CI environment without a full site running, making true continuous integration difficult.
In short, WordPress’s out-of-the-box tooling assumes a manual, one-server setup, which conflicts with the declarative, automated approach that DevOps teams prefer.

The cultural side of the WordPress community also hasn’t emphasized DevOps historically. Many WordPress developers come from content or design backgrounds, not traditional software engineering, and may not be familiar with practices like infrastructure as code, containerization, or continuous delivery, and this is normal.

As a result, a lot of WordPress development happens in “silos”,  a developer might work in isolation on code, a sysadmin (if one exists) manages servers separately, and there’s minimal collaboration or automation bridging the two.

Why This Matters: Risks of Sticking to Traditional Approaches

Why do these DevOps shortcomings in the WordPress world matter so much? Simply put, a lack of modern infrastructure practices can hurt both the performance and the profitability of a WordPress-based business. Let’s break down some of the concrete impacts:

  • Scalability and Uptime: WordPress sites without DevOps-driven infrastructure often run on a single server or a handful of overprovisioned servers with no auto-scaling. This might work fine during normal days, but what about traffic surges?

    If a blog post goes viral or an e-commerce site has a flash sale, a non-scalable setup will either crash or require massive idle resources to handle the peak. Many small WordPress sites live in fear of sudden popularity,  a formerly “little” site that suddenly gets thousands of visitors can simply go down, because it wasn’t designed to scale out.
  • Overprovisioning and Cost: To avoid crashes, many WordPress site owners take the brute-force approach: throw more hardware at the problem “just in case.” It’s not uncommon to see a WooCommerce site deployed on a huge single server with vastly more CPU/RAM than it needs most of the time  essentially using a sledgehammer to crack a nut.

    For example, one website owner was running ~100 separate WordPress sites on a single monster server (16 CPU cores, 64 GB RAM) costing around $350–400 per month. Most of the time, this beefy server was underutilized, but whenever one of the sites had a traffic spike, it hogged resources and knocked the other sites offline. This scenario is more common than you’d think: without DevOps guidance, people tend to either overprovision a single environment or spin up dozens of tiny servers per site, which in that real case would have been even more expensive if done naively.

    Reactive scaling (upgrading to a bigger host only after an outage, or paying for large instances “just in case”) is a costly and inefficient strategy. In contrast, a well-planned DevOps approach emphasizes proactive, right-sized infrastructure, using automation to add capacity when needed and roll it back when not, which can save huge amounts of money over time. Without those practices, businesses often overspend on hosting or suffer expensive downtime that a smarter setup could have prevented.
  • Maintenance and Deployment Pain: A WordPress site of any significant size is a living system, with core updates every few months, a constant stream of plugin updates, theme changes, content additions, etc.

    In a non-DevOps setup, applying these updates can be a nightmare. If you have to update plugins on a production site manually, you risk introducing errors that take the site down.

    If you don’t update (for fear of breaking something), you accumulate technical debt and security vulnerabilities. It’s a catch-22 that modern DevOps workflows are designed to solve via staging environments, automated testing, and continuous deployment.

    Many WordPress teams still lack these safety nets. In the WordPress community forum, developers note that plenty of sites operate with “inexperienced devs, tight deadlines, and clients unwilling to pay for quality,” resulting in hasty changes on live sites and little time for proper testing. The result is often a cycle of break-fix patches, plugins added to band-aid issues, and mounting instability.

    This “works on my machine” syndrome is exactly what DevOps culture tries to eliminate by introducing automated testing, and collaboration between development and operations. Without these, WordPress site owners are essentially flying without instruments,  every update is a gamble that could incur hours of firefighting if things go wrong.

In summary, sticking to the traditional, ad-hoc way of managing WordPress might seem easier initially, but it exacts a toll in scalability, reliability, and cost efficiency as a site grows. The bigger and more complex your WordPress site (especially for high-traffic blogs or busy WooCommerce stores), the more painful and risky it becomes to operate without DevOps best practices in place. Next, let’s look at how embracing DevOps can turn these challenges into opportunities.

Bringing DevOps to WordPress: What Does it Entail?

“DevOps” may sound like a buzzword, but at its core it’s about bridging the gap between development and operations through better process and automation.

In a WordPress context, adopting DevOps means shifting away from one-off manual work towards streamlined, repeatable workflows for building, testing, and deploying your site.

It’s both a cultural change (developers, IT admins, and content teams collaborating more closely) and a technical change (using tools to automate tasks and manage infrastructure). Here are some key DevOps practices and how they can be applied to WordPress:

  • Infrastructure management with terraform-ansible: In a case where scaling is crucial, many things could go wrong, If an e-commerce site for example, does not implement automated processes in order to have a consistent infrastructure.

    A usual case could be a promotion campaign that a retailer may run, so the need for server capacity is growing instantly for like 2 days only. Imagine how much this store can improve the infrastructure robustness even with a more traditional way, with no container and orchestrators in place.

    i.e an architecture that combines terraform(for servers creation and cloud native sources binding), ansible (for servers provisioning) and bash scripts (for automated processes that may need to run on each server, or the new server cluster that is created).
  • Automated Updates and Maintenance: WordPress’s constant stream of core and plugin updates can be a double-edged sword,  great for security and improvements, but onerous to keep up with manually. DevOps practices encourage automation of these repetitive maintenance tasks.

    For example, you might use a tool or script to automatically apply updates in a staging environment nightly, run tests, and then push them to production if all looks good. At scale, WordPress operators also use configuration management tools (like Ansible, Puppet, or Chef) to enforce consistent setups across many servers or sites,  so you don’t end up with snowflake servers that each have to be fixed individually. Automated backups, database migrations, and monitoring scripts also fall under this umbrella.

    The goal is that routine operations are not dependent on someone clicking buttons; they happen reliably via code.
  • Environment Consistency (Dev/Staging/Prod): A hallmark of DevOps is maintaining multiple environments (development, staging, production) that mirror each other, so that you can test changes in a safe sandbox before affecting users. In WordPress, setting up a staging site is increasingly considered a best practice, but many smaller operations still lack one.

    Modern WordPress hosting providers and DevOps-oriented agencies insist on this workflow: developers push code to a staging environment identical to production, run all necessary checks, and only then apply the changes to the live site. This eliminates the old “it works on my machine” problem, because your machine (or container) is configured the same as the live servers.
  • Collaboration Between Teams: DevOps is not just tools, it’s also people. In a WordPress project adopting DevOps, developers, designers, content editors, and sysadmins all work more in tandem rather than in isolation.

    This might mean implementing agile practices (daily standups where developers and operations discuss ongoing issues), using communication tools to keep everyone on the same page, and fostering a culture where developers think about operational impacts (performance, security) and ops folks are involved early in development decisions.

    For a business owner or agency, this cross-team collaboration translates to faster problem resolution and more innovative solutions,  because the people building the site and those maintaining it are sharing knowledge and responsibility.

    A simple example: If a marketing team plans a huge campaign that might spike traffic, in a DevOps culture they’d loop in the IT team ahead of time to prepare capacity. In a siloed culture, they might not, and the site could crash. Breaking down these silos is thus critical for high-traffic WordPress operations.

In essence, DevOps for WordPress means applying the same rigor and automation that big tech companies use for their software, but tailored to the WordPress ecosystem. This might involve some upfront effort, adopting new tools and workflows, but the payoff is enormous for sites that have a lot at stake. The next sections will delve into one of the most critical areas that DevOps can address for WordPress sites: scalability of the hosting infrastructure.

Scaling WordPress: From Overprovisioned Servers to Cloud Native

One of the clearest benefits of bringing DevOps practices to WordPress is unlocking true scalability, the ability for your site’s infrastructure to grow (or shrink) on demand to match your traffic. Traditionally, many WordPress sites started on a single LAMP server (Linux, Apache, MySQL, PHP) and stayed that way. But as we’ve discussed, a single server has finite limits and can become a single point of failure. Let’s look at how modern infrastructure (often cloud-based and automated) can transform WordPress performance for high traffic scenarios.

Vertical vs. Horizontal Scaling: Scaling essentially comes in two flavors. Vertical scaling (or “scaling up”) means you upgrade your server  e.g. move from a 4 CPU / 8 GB RAM machine to an 16 CPU / 64 GB RAM machine. Horizontal scaling (or “scaling out”) means you add more servers to handle traffic, distributing requests among them. In the context of WordPress, vertical scaling is straightforward (most hosting providers let you switch to a bigger box easily) and can address moderate growth, but it hits diminishing returns, as there’s a limit to how big one database or web server can get.

Horizontal scaling is far more powerful for large scale: by running multiple WordPress servers behind a load balancer, your site can serve many more users and also gain redundancy (if one server crashes, others can pick up the slack).

In fact, without horizontal scaling, “any WordPress website hosted in a standard environment… simply won’t cope with the demand of high traffic” beyond a certain point.

The largest WordPress-powered sites (like an e-commerce website during a marketing campaign) all use horizontal scaling, with clusters of web servers and database replicas working in tandem.

Modern Cloud Infrastructure: Enter the world of cloud and containers. DevOps for WordPress often means moving away from pets to cattle  (from one special server you coddle to a fleet of servers you can replace or scale at will).

Cloud providers allow you to script your infrastructure so you can deploy multiple WordPress instances with a few API calls or auto-scaling rules. Even more cutting-edge is using containerization and orchestration  for example, packaging WordPress in a Docker container and running it on Kubernetes or Docker Swarm.

This might sound complex, but it’s increasingly common and for good reason. Running WordPress on Kubernetes allows you to leverage automated scaling, self-healing, and managed deployments.

As one tech consultancy put it, “Leveraging Kubernetes for deploying WordPress can significantly enhance its scalability and availability, providing a robust solution for high-traffic websites.”

In Kubernetes, you can define a deployment for WordPress that says “always run X copies of this app, scale out to Y copies if CPU usage goes above 80%,” and the platform will automatically add pods (containers) of WordPress to handle surges.

You also get easy strategies for high availability: for instance, you can run your cluster across multiple data centers so that even if one goes down, your site stays up.

To ground this in a real example, consider a WooCommerce store preparing for Black Friday traffic. Rather than renting a giant server that sits mostly idle until the big day, the DevOps approach would be to containerize the WordPress app and configure auto-scaling. When traffic starts ramping up, the system might spin up additional WordPress containers (each running the same code) and perhaps additional database read-replicas to spread out the query load.

A load balancer in front would distribute incoming shoppers to different instances. The result is that your store could handle 10x, 100x traffic when needed, but you’re not permanently paying for 100x resources when traffic is normal. This elasticity is a core benefit of cloud DevOps architectures.

Caching, CDN, and Other Enhancements: It’s worth noting that scaling isn’t only about adding servers; it’s also about reducing load per server. DevOps teams ensure that WordPress installations make heavy use of caching layers and CDNs (Content Delivery Networks) to optimize performance.

At the server level, object caching (with Redis or Memcached) and full-page caching can drastically cut down CPU load by serving frequent requests from memory. Many WordPress plugins offer caching, but a DevOps approach might implement it at the server or proxy level for more efficiency.

Database Scaling: For a dynamic site (especially WooCommerce stores with tons of products and orders), the database can become a bottleneck. DevOps solutions include database replication and optimization. You might have a master database for writes and multiple read replicas for heavy read traffic, all kept in sync.

Load balancers (or WordPress plugins/configs) can direct read queries to replicas. Also, careful indexing and query optimization are needed, something a DevOps database administrator would continuously work on.

Modern WordPress at scale even supports separating the database for certain data, such as using external search services (ElasticSearch) for product searches rather than taxing MySQL, or using WooCommerce’s High-Performance Order Storage (HPOS) to split order data into separate tables for efficiency.

The main point is, scaling WordPress is a multi-faceted exercise: you scale the web servers horizontally, beef up or replicate the database, and add caching/CDN layers. Together these approaches can handle enormous traffic loads that no single server setup could manage.

All of the above might sound complex to a small team, but you don’t necessarily have to build it all from scratch yourself. There’s an entire ecosystem of WordPress-focused DevOps and hosting services that provide these capabilities. 

The post Rethinking WordPress Infrastructure: The DevOps Imperative appeared first on Cloudwork.

]]>
Benchmarking Hetzner’s Storage Box: Speed, Use Cases & Real World Performance /benchmarking-hetzners-storage-box-speed-use-cases-real-world-performance/ Tue, 01 Jul 2025 17:00:18 +0000 /?p=965 What is Hetzner’s Storage Box? Hetzner’s Storage Box is a self managed, RAID backed online storage solution that just became available in the new Hetzner Cloud Console, making provisioning fast and streamlined. It’s built for: Real World Benchmarks: Upload & I/O Performance Here are the results from our tests against a Storage Box in Germany, Falkenstein. The benchmarks were executed on a CPX11 Hetzner cloud instance on the same region: Test Throughput Duration SFTP: Upload 100 MB 74.2 MB/s 1.41 s SFTP: Upload 1 GB 72.6 MB/s 14.17 s SFTP: Upload 10 GB 71.7 MB/s 2 m 22.8 s Local dd dsync write (with options bs=1M count=1024, total of 1GB) 12.7 MB/s 84.8 s Local dd read (1GB) 75.8 MB/s 14.17 s lftp mirror (100 × 10 MB) 71.8 MB/s 14 s Notes: Key takeaways: Who Are Hetzner Storage Boxes For? Hetzner Storage Boxes are a great fit for developers, sysadmins, and teams who need reliable, protocol flexible, and cost effective storage. Here’s who will benefit most: If you don’t need a web UI but do need fast, stable, scriptable storage—then Hetzner Storage Boxes are a smart and scalable choice. To explore more cloud storage options and their trade offs, read our comparison on S3 Standard vs S3 Express One Zone.

The post Benchmarking Hetzner’s Storage Box: Speed, Use Cases & Real World Performance appeared first on Cloudwork.

]]>
What is Hetzner’s Storage Box?

Hetzner’s Storage Box is a self managed, RAID backed online storage solution that just became available in the new Hetzner Cloud Console, making provisioning fast and streamlined.
It’s built for:

  • Flexible data access via SMB, WebDAV and SSH protocols, as well as all transfer protocols based on those 3
  • Reliable storage: RAID arrays with checksums for disk‑failure resilience and data integrity
  • Unlimited traffic, up/down, and upscale/downscale flexibility with no longterm contract
  • Snapshots support: manual and scheduled automated snapshots (10–40 slots depending on plan)
  • Multiple users: up to 100 sub accounts, each restricted to a subdirectory with main user oversight

Real World Benchmarks: Upload & I/O Performance

Here are the results from our tests against a Storage Box in Germany, Falkenstein.

The benchmarks were executed on a CPX11 Hetzner cloud instance on the same region:

TestThroughputDuration
SFTP: Upload 100 MB74.2 MB/s1.41 s
SFTP: Upload 1 GB72.6 MB/s14.17 s
SFTP: Upload 10 GB71.7 MB/s2 m 22.8 s
Local dd dsync write (with options bs=1M count=1024, total of 1GB)12.7 MB/s84.8 s
Local dd read (1GB)75.8 MB/s14.17 s
lftp mirror (100 × 10 MB)71.8 MB/s14 s

Notes:

  • sshfs was used to mount the Hetzner’s Storage Box using SFTP
  • lftp was used to test parallel file uploads, again using the SFTP protocol

Key takeaways:

  • SFTP upload consistently achieves ~70–75 MB/s, even for large transfers
  • Excellent Protocol Flexibility, SFTP, FTPS, SMB, WebDAV, Rsync, Rclone, making it easy to integrate into various backup tools
  • Snapshot & Multi User Support
  • Cost effective, starting from 3.20 € per month for 1TB of storage
  • Now Available in the Cloud Console. Hetzner’s Storage Boxes can now be provisioned directly via the Hetzner Cloud Console, making setup easier and faster than before. A Github issue is also created to adopt this new resource on Hetzner’s terraform provider!

Who Are Hetzner Storage Boxes For?

Hetzner Storage Boxes are a great fit for developers, sysadmins, and teams who need reliable, protocol flexible, and cost effective storage.

Here’s who will benefit most:

  • Backup and archival users: Ideal for automated offsite backups using tools like Rsync, Veeam, Restic, Borg, or Rclone
  • Media professionals and researchers: Handle large datasets (e.g. media archives, research dumps) with predictable upload/download performance
  • Small teams or agencies: With up to 100 sub accounts, each team or client can have isolated access to their own storage space
  • Anyone in need for remote storage without vendor lock in: Use open protocols like SFTP, SMB, WebDAV, and avoid proprietary APIs


If you don’t need a web UI but do need fast, stable, scriptable storage—then Hetzner Storage Boxes are a smart and scalable choice.

To explore more cloud storage options and their trade offs, read our comparison on S3 Standard vs S3 Express One Zone.

The post Benchmarking Hetzner’s Storage Box: Speed, Use Cases & Real World Performance appeared first on Cloudwork.

]]>
The True Depth of Model Context Protocol (MCP) /model-context-protocol-not-just-a-glorified-rest-api-wrapper/ Fri, 13 Jun 2025 14:54:39 +0000 /?p=926 The True Depth of Model Context Protocol (MCP) The True Depth of MCP: Beyond Just Tools The Model Context Protocol (MCP) has gained substantial attention in the AI development community, not just for its promise of interoperability but also for its potential in agentic architectures. Many practitioners though, reduce MCP to a simple tool execution layer, an oversimplification that masks its true depth. The Surface: Tools Are Just the Beginning Most developers currently focus on the “tools” component of MCP protocol, which enables interaction with APIs, such as querying databases, sending messages via Slack, or retrieving user information. Not to say that those tools are not useful. They mimic RESTful API capabilities but through a language model-aware protocol. However, judging MCP only by this layer is like evaluating a smartphone solely by its calculator app!! The Full Scope of Model Context Protocol MCP: Five Core Capabilities To further illuminate these capabilities, we will examine some real-world examples and their practical significance: MCP protocol defines five core capabilities: Tools: API wrappers to perform actions.Resources: Shared data elements (e.g., files, URIs) between clients and servers.Prompts: Parameterized templates for generating LLM.Sampling: Delegates LLM generation to clients, allowing contextual synthesis and human-in-the-loop workflows.Roots: Define operational boundaries and constraints for data and actions. 🧩 Tools: Operational Action Executors Example: A Slack MCP server allows an AI agent to list channels, query messages, or send DMs.Use Case: Ideal for workflow automation bots that interact with teams across communication platforms. Allow easy integration with productivity tools. 📁 Resources: Contextual Anchors for Data Exchange Resources in MCP are abstract definitions of data elements that clients and servers can exchange. For instance, clients can provide local files or database object references like files: or postgres: URIs. These references allow the server to work with specific, predefined data sets, ensuring consistency and efficiency with no manual configuration. 📝 Prompts: Server-Side Intelligence, Client-Side Power Example: A legal department deploys an MCP server that provides templated legal document drafts. A prompt might be “Generate an NDA agreement for [Company A] and [Company B] for a term of [X] months.”Use Case: Every client in the company (web, mobile, CLI) can use this standard template, eliminating manual drafting and reducing errors. Prompts are server-side features that deliver parameterized templates to clients. If a server hosts these templates, they can be reused across multiple clients, ensuring consistent, high-quality LLM outputs without custom prompts every time. 🖥️ The Client Side: More Than Just a Caller MCP also empowers the client to do more than just relay calls.   ✨ Sampling: The Magic of Local LLM Generation Example: A research tool aggregates data from multiple databases via MCP servers and synthesizes a summary using a local LLM.Use Case: Maintains privacy, allow custom context, and supports review loops. Sampling enables the client to generate LLM responses based on collected data, without relying on external processing. 🌱 Roots: Clear-Cut Operational Boundaries Example: A code assistant sets the MCP root to a specific folder.Use Case: Ensures the server doesn’t access files outside the designated scope. Roots define where and what the server is allowed to touch, by safeguarding data boundaries and protecting user trust.     🧠 A Real-Life Workflow That Combines All Five MCP Capabilities Let’s put it all together with a practical scenario:Imagine a software company wants to automate the process of generating daily engineering reports using an AI-powered assistant. Here’s how each part of MCP makes the workflow both powerful and safe: 🧩 Tools – Getting the Right Information What happens:The assistant uses built in connections (tools) to fetch data from services like Slack (to see what was discussed) and PostgreSQL (to get information from databases). Simple Example:The AI bot pulls messages from the team’s Slack channel to see what’s been worked on, and checks the database for new code changes. Why this matters:Without tools, the assistant couldn’t gather all the information needed for a useful report.   📁 Resources – Pointing to the Right Data What happens:Instead of uploading or copying files everywhere, the AI can refer directly to important files, like code logs or change reports, using file paths or database links. Simple Example:It might use a direct link to the code changes made today, or point to a log file from a database, rather than copying everything around. Why this matters:This avoids confusion about “which file” or “which data” to use. Everyone is on the same page, and the assistant always knows what’s current.   📝 Prompts – Using Ready-Made Templates What happens:The server gives the assistant a template for the report, like a fill-in-the-blanks form with fields for things like [file_group] or [channel]. Simple Example:Instead of writing the report from scratch every day, the assistant fills out a report template:“Today, changes were made in [file_group] and discussed in [channel].” Why this matters:This ensures every report is structured the same way, is easy to read, and nothing is accidentally missed.   ✨ Sampling – Local Report Generation for Privacy and Customization What happens:The assistant writes the report locally using its own AI, combining the data it collected. This can happen on the company’s computers, not sent to a third party. Simple Example:The report draft is created on your machine, not in the cloud, so sensitive info stays inside the company. Why this matters:Keeps private data safe, lets the team review or edit before sending, and can even tailor the report using company-specific language.   🌱 Roots – Setting Boundaries for Security What happens:The system is told exactly where it’s allowed to look and what it can touch like which folders or Slack channels. It can’t wander off and access things it shouldn’t. Simple Example:The assistant only looks at code in the current “Sprint” folder and messages in the #DevOps Slack channel, not in private messages or HR files. Why this matters:Protects sensitive info, respects privacy with no surprises or accidental leaks.     Putting it all together Without tools, you can’t gather info.On top of that, without resources, you don’t know which data

The post The True Depth of Model Context Protocol (MCP) appeared first on Cloudwork.

]]>
Cloudwork cover page

The True Depth of Model Context Protocol (MCP)

The True Depth of MCP: Beyond Just Tools

The Model Context Protocol (MCP) has gained substantial attention in the AI development community, not just for its promise of interoperability but also for its potential in agentic architectures.
Many practitioners though, reduce MCP to a simple tool execution layer, an oversimplification that masks its true depth.

The Surface: Tools Are Just the Beginning

Most developers currently focus on the “tools” component of MCP protocol, which enables interaction with APIs, such as querying databases, sending messages via Slack, or retrieving user information. Not to say that those tools are not useful.
They mimic RESTful API capabilities but through a language model-aware protocol. However, judging MCP only by this layer is like evaluating a smartphone solely by its calculator app!!

The Full Scope of Model Context Protocol MCP: Five Core Capabilities

To further illuminate these capabilities, we will examine some real-world examples and their practical significance:

MCP protocol defines five core capabilities:

Tools: API wrappers to perform actions.
Resources: Shared data elements (e.g., files, URIs) between clients and servers.
Prompts: Parameterized templates for generating LLM.
Sampling: Delegates LLM generation to clients, allowing contextual synthesis and human-in-the-loop workflows.
Roots: Define operational boundaries and constraints for data and actions.

🧩 Tools: Operational Action Executors

Example: A Slack MCP server allows an AI agent to list channels, query messages, or send DMs.
Use Case: Ideal for workflow automation bots that interact with teams across communication platforms. Allow easy integration with productivity tools.

📁 Resources: Contextual Anchors for Data Exchange

Resources in MCP are abstract definitions of data elements that clients and servers can exchange. For instance, clients can provide local files or database object references like files: or postgres: URIs.
These references allow the server to work with specific, predefined data sets, ensuring consistency and efficiency with no manual configuration.

📝 Prompts: Server-Side Intelligence, Client-Side Power

Example: A legal department deploys an MCP server that provides templated legal document drafts. A prompt might be “Generate an NDA agreement for [Company A] and [Company B] for a term of [X] months.”
Use Case: Every client in the company (web, mobile, CLI) can use this standard template, eliminating manual drafting and reducing errors.

Prompts are server-side features that deliver parameterized templates to clients. If a server hosts these templates, they can be reused across multiple clients, ensuring consistent, high-quality LLM outputs without custom prompts every time.

🖥 The Client Side: More Than Just a Caller

MCP also empowers the client to do more than just relay calls.

 

✨ Sampling: The Magic of Local LLM Generation

Example: A research tool aggregates data from multiple databases via MCP servers and synthesizes a summary using a local LLM.
Use Case: Maintains privacy, allow custom context, and supports review loops.

Sampling enables the client to generate LLM responses based on collected data, without relying on external processing.

🌱 Roots: Clear-Cut Operational Boundaries

Example: A code assistant sets the MCP root to a specific folder.
Use Case: Ensures the server doesn’t access files outside the designated scope.

Roots define where and what the server is allowed to touch, by safeguarding data boundaries and protecting user trust.

 


 

🧠 A Real-Life Workflow That Combines All Five MCP Capabilities

Let’s put it all together with a practical scenario:
Imagine a software company wants to automate the process of generating daily engineering reports using an AI-powered assistant.
Here’s how each part of MCP makes the workflow both powerful and safe:

🧩 Tools – Getting the Right Information

What happens:
The assistant uses built in connections (tools) to fetch data from services like Slack (to see what was discussed) and PostgreSQL (to get information from databases).

Simple Example:
The AI bot pulls messages from the team’s Slack channel to see what’s been worked on, and checks the database for new code changes.

Why this matters:
Without tools, the assistant couldn’t gather all the information needed for a useful report.

 

📁 Resources – Pointing to the Right Data

What happens:
Instead of uploading or copying files everywhere, the AI can refer directly to important files, like code logs or change reports, using file paths or database links.

Simple Example:
It might use a direct link to the code changes made today, or point to a log file from a database, rather than copying everything around.

Why this matters:
This avoids confusion about “which file” or “which data” to use. Everyone is on the same page, and the assistant always knows what’s current.

 

📝 Prompts – Using Ready-Made Templates

What happens:
The server gives the assistant a template for the report, like a fill-in-the-blanks form with fields for things like [file_group] or [channel].

Simple Example:
Instead of writing the report from scratch every day, the assistant fills out a report template:
“Today, changes were made in [file_group] and discussed in [channel].”

Why this matters:
This ensures every report is structured the same way, is easy to read, and nothing is accidentally missed.

 

✨ Sampling – Local Report Generation for Privacy and Customization

What happens:
The assistant writes the report locally using its own AI, combining the data it collected. This can happen on the company’s computers, not sent to a third party.

Simple Example:
The report draft is created on your machine, not in the cloud, so sensitive info stays inside the company.

Why this matters:
Keeps private data safe, lets the team review or edit before sending, and can even tailor the report using company-specific language.

 

🌱 Roots – Setting Boundaries for Security

What happens:
The system is told exactly where it’s allowed to look and what it can touch like which folders or Slack channels. It can’t wander off and access things it shouldn’t.

Simple Example:
The assistant only looks at code in the current “Sprint” folder and messages in the #DevOps Slack channel, not in private messages or HR files.

Why this matters:
Protects sensitive info, respects privacy with no surprises or accidental leaks.

 

 

Putting it all together

Without tools, you can’t gather info.
On top of that, without resources, you don’t know which data to use.
Additionally without prompts, you get messy, inconsistent reports.
Without sampling, you lose privacy and flexibility.
At last, without roots, you risk leaking or messing with the wrong data.

All five features combine to give you automation that’s smart, safe, and genuinely useful, not just a fancy API call!

 

The community gap

Despite this richness, unfortunately most client implementations today only support tools.
As a result, this has created a gap in understanding and utility.
Features like prompt retrieval or sampling are often buried in poor UI/UX, making them hard to discover or test. Consequently, the full promise of MCP remains untapped, but the optimistic part is that
as awareness grows and implementations improve, we can expect these capabilities to become more accessible and meaningful for the AI community!!

 

The post The True Depth of Model Context Protocol (MCP) appeared first on Cloudwork.

]]>
S3 Standard vs S3 Express One Zone: Which One Should You Choose? /s3-standard-vs-s3-express-one-zone/ Mon, 19 May 2025 14:03:20 +0000 /?p=889 S3 Standard vs S3 Express One Zone: Which One Should You Choose? Amazon S3 (Simple Storage Service) offers several storage classes tailored to different performance, availability, and cost needs. In this post, we’ll compare S3 Standard vs S3 Express One Zone, two popular options with very different performance profiles, to help you decide which one is right for your workload. What Is S3 Standard? S3 Standard is the default storage class in Amazon S3. It’s designed for frequently accessed data and offers high durability (99.999999999%), availability across multiple Availability Zones (AZs), and millisecond latency for most operations. It’s a solid general-purpose choice for most workloads that require consistent availability and strong resilience. What Is S3 Express One Zone? S3 Express One Zone is a newer storage class optimized for ultra-low latency and high throughput. Unlike S3 Standard, it stores data in a single AZ, which significantly reduces network hops—resulting in much faster response times. While S3 Express sacrifices cross-AZ redundancy, it’s ideal for performance-sensitive applications such as AI/ML pipelines, real-time data processing, and analytics workloads. Performance Comparison: S3 Standard vs S3 Express One Zone We benchmarked common upload and download operations with object sizes ranging from 4KB to 512KB. The results show dramatic latency reductions when using S3 Express One Zone, especially when accessed from the same Availability Zone (AZ). Key Findings Download operations saw a 70–90% reduction in latency. For example, downloading a 4KB object dropped from an average of 19 ms (S3 Standard) to just 3.8 ms with S3 Express—an 80% improvement. Upload operations improved even more. Uploading a 128KB object averaged 53 ms with S3 Standard and only 5.5 ms with S3 Express—an almost 90% decrease in latency. Even when accessed from a different AZ within the same region, S3 Express still delivered significantly better performance—typically 60–85% faster than S3 Standard. Note: All performance tests were executed from an Amazon EC2 instance within the same AWS region as the S3 buckets. This setup ensured consistent network conditions and reflects typical access patterns. Key Takeaways When Should You Use S3 Express One Zone? Choose S3 Express One Zone if: Choose S3 Standard if: Final Thoughts S3 Express One Zone is a game-changer for building super low-latency, high-performance cloud apps—but it’s not meant to replace S3 Standard in every situation. It really depends on what your workload needs. If speed is your top priority and you’re comfortable designing around a single Availability Zone, S3 Express is a strong choice. But if you’re looking for maximum availability and don’t want to think about AZ-specific architecture, S3 Standard is still the go-to option. Just keep in mind that S3 Express One Zone isn’t available in every region yet. You can check the list of supported regions here.

The post S3 Standard vs S3 Express One Zone: Which One Should You Choose? appeared first on Cloudwork.

]]>
S3 Standard vs S3 Express One Zone: Which One Should You Choose?
S3 Standard vs S3 Express One Zone: Which AWS Storage Class Should You Use?

Amazon S3 (Simple Storage Service) offers several storage classes tailored to different performance, availability, and cost needs.

In this post, we’ll compare S3 Standard vs S3 Express One Zone, two popular options with very different performance profiles, to help you decide which one is right for your workload.


What Is S3 Standard?

S3 Standard is the default storage class in Amazon S3. It’s designed for frequently accessed data and offers high durability (99.999999999%), availability across multiple Availability Zones (AZs), and millisecond latency for most operations.

It’s a solid general-purpose choice for most workloads that require consistent availability and strong resilience.


What Is S3 Express One Zone?

S3 Express One Zone is a newer storage class optimized for ultra-low latency and high throughput. Unlike S3 Standard, it stores data in a single AZ, which significantly reduces network hops—resulting in much faster response times.

While S3 Express sacrifices cross-AZ redundancy, it’s ideal for performance-sensitive applications such as AI/ML pipelines, real-time data processing, and analytics workloads.


Performance Comparison: S3 Standard vs S3 Express One Zone

We benchmarked common upload and download operations with object sizes ranging from 4KB to 512KB. The results show dramatic latency reductions when using S3 Express One Zone, especially when accessed from the same Availability Zone (AZ).

Key Findings

Download operations saw a 70–90% reduction in latency. For example, downloading a 4KB object dropped from an average of 19 ms (S3 Standard) to just 3.8 ms with S3 Express—an 80% improvement.

Upload operations improved even more. Uploading a 128KB object averaged 53 ms with S3 Standard and only 5.5 ms with S3 Express—an almost 90% decrease in latency.

Even when accessed from a different AZ within the same region, S3 Express still delivered significantly better performance—typically 60–85% faster than S3 Standard.

Note: All performance tests were executed from an Amazon EC2 instance within the same AWS region as the S3 buckets. This setup ensured consistent network conditions and reflects typical access patterns.


Key Takeaways

  • S3 Express is 4–10x faster than S3 Standard for both upload and download operations, especially with smaller objects.
  • Latency is lowest when accessing S3 Express from the same Availability Zone, though performance remains excellent even across AZs in the same region.
  • S3 Standard offers higher availability and durability, suitable for general-purpose storage needs.
  • S3 Express One Zone is ideal for latency-critical workloads that can tolerate single-AZ storage.

When Should You Use S3 Express One Zone?

Choose S3 Express One Zone if:

  • You need ultra-fast performance, especially for small object access.
  • Your application is latency-sensitive, such as real-time data processing or ML training.
  • You can handle the trade-off of storing data in a single AZ (or replicate it manually if needed).

Choose S3 Standard if:

  • Your priority is high availability and durability across AZs.
  • Your workload is not especially latency-sensitive.
  • You want a simpler, fully managed solution without worrying about zone-level architecture.

Final Thoughts

S3 Express One Zone is a game-changer for building super low-latency, high-performance cloud apps—but it’s not meant to replace S3 Standard in every situation. It really depends on what your workload needs.

If speed is your top priority and you’re comfortable designing around a single Availability Zone, S3 Express is a strong choice. But if you’re looking for maximum availability and don’t want to think about AZ-specific architecture, S3 Standard is still the go-to option.

Just keep in mind that S3 Express One Zone isn’t available in every region yet. You can check the list of supported regions here.

The post S3 Standard vs S3 Express One Zone: Which One Should You Choose? appeared first on Cloudwork.

]]>
Claude Code: The best terminal AI Agent for software engineers /claude-code-the-best-terminal-ai-agent-for-software-engineers/ Sat, 17 May 2025 15:04:51 +0000 /?p=884  Introduction  When it comes to software engineering , whether you’re deep into application development, infrastructure operations, or platform engineering , having an AI assistant that lives in your workflow can be a game-changer. The last days I experiment with Claude Code, a terminal-based AI agent from Anthropic, designed to run alongside  your favorite IDEs without being tied to any of them.In this post, I’ll share my initial impressions, highlight its standout features, point out its disadvantages, and provide some practical tips to make the most of it.  Why Terminal-First Matters  Most AI code assistants today are IDE extensions; they’re great, but they often force you to context-switch. Claude Code breaks that mold:   Always at hand: Runs directly in your terminal, so you don’t have to leave your shell. Universal compatibility: Works with any IDE or editor you prefer like VS Code, Vim, JetBrains, you name it. Workflow continuity: Whether you’re SSH’d into a server, running CI pipelines, or editing code locally, Claude Code is right there.  Powered by the Anthropic API  Claude Code uses the Anthropic API under the hood. While it’s not the cheapest ,pricing is roughly 10× that of some alternatives like Google’s Gemini or DeepSeek, Anthropic offers some nice methods that we will mention in a while, and can help you track your costs effectively  Who’s It For?  DevOps, SREs, platform engineers and software devs all get boosted productivity ,from shell hacks and CI/CD snippets to infra-as-code scaffolding and instant code help, though you’ll still need an experienced engineer to keep things on point.  Key Features & Commands  Below are some of the most useful initial commands that you can utilize : Command Purpose /init Scans your current directory, reads project context, and generates a Markdown summary (.md) for reference. /cost Displays the total spend (in USD) for the ongoing session. # {request that you want to remember} You can pass parameters for your preferences—for example, “Every time I ask you to open a document, open it in VS Code.” You can then save this preference in the project and commit it to the repo so the whole team can use it.  Pricing & Billing Transparency  One concern with Claude Code is cost. It uses a token-based payment system, but if you apply the next two principles, you’ll always stay on top of your bill:   Prepaid model: Add any amount you’re comfortable with (e.g., $5, $10) and top up as needed. Real-time insights: /cost in-terminal or check the Billing Dashboard to avoid surprises. Despite the premium pricing, the accuracy, context-awareness, and terminal integration make it an exceptional tool for mission-critical workflows.  Final Thoughts Claude Code currently represents the best AI agent for software engineering, particularly if your daily workflow revolves around the terminal. Its seamless integration, powerful context handling, and pay-as-you-go flexibility set it apart. While AI evolves rapidly and alternatives may emerge, for now, Claude Code is my go-to tool for DevOps, SRE, and platform tasks. Internal Links for Further Reading: RDS advanced configuration and monitoring

The post Claude Code: The best terminal AI Agent for software engineers appeared first on Cloudwork.

]]>

 Introduction

 When it comes to software engineering , whether you’re deep into application development, infrastructure operations, or platform engineering , having an AI assistant that lives in your workflow can be a game-changer. The last days I experiment with Claude Code, a terminal-based AI agent from Anthropic, designed to run alongside  your favorite IDEs without being tied to any of them.In this post, I’ll share my initial impressions, highlight its standout features, point out its disadvantages, and provide some practical tips to make the most of it.

 Why Terminal-First Matters

 Most AI code assistants today are IDE extensions; they’re great, but they often force you to context-switch. Claude Code breaks that mold:

 

    • Always at hand: Runs directly in your terminal, so you don’t have to leave your shell.

    • Universal compatibility: Works with any IDE or editor you prefer like VS Code, Vim, JetBrains, you name it.

    • Workflow continuity: Whether you’re SSH’d into a server, running CI pipelines, or editing code locally, Claude Code is right there.

 Powered by the Anthropic API

 Claude Code uses the Anthropic API under the hood. While it’s not the cheapest ,pricing is roughly 10× that of some alternatives like Google’s Gemini or DeepSeek, Anthropic offers some nice methods that we will mention in a while, and can help you track your costs effectively

 Who’s It For?
 DevOps, SREs, platform engineers and software devs all get boosted productivity ,from shell hacks and CI/CD snippets to infra-as-code scaffolding and instant code help, though you’ll still need an experienced engineer to keep things on point.

 Key Features & Commands

 Below are some of the most useful initial commands that you can utilize :

Command Purpose
/init Scans your current directory, reads project context, and generates a Markdown summary (.md) for reference.
/cost Displays the total spend (in USD) for the ongoing session.
# {request that you want to remember} You can pass parameters for your preferences—for example, “Every time I ask you to open a document, open it in VS Code.” You can then save this preference in the project and commit it to the repo so the whole team can use it.

 Pricing & Billing Transparency

 One concern with Claude Code is cost. It uses a token-based payment system, but if you apply the next two principles, you’ll always stay on top of your bill:

 

    1. Prepaid model: Add any amount you’re comfortable with (e.g., $5, $10) and top up as needed.

    1. Real-time insights: /cost in-terminal or check the Billing Dashboard to avoid surprises.

Despite the premium pricing, the accuracy, context-awareness, and terminal integration make it an exceptional tool for mission-critical workflows.

 Final Thoughts

Claude Code currently represents the best AI agent for software engineering, particularly if your daily workflow revolves around the terminal.
Its seamless integration, powerful context handling, and pay-as-you-go flexibility set it apart.

While AI evolves rapidly and alternatives may emerge, for now, Claude Code is my go-to tool for DevOps, SRE, and platform tasks.



Internal Links for Further Reading:

RDS advanced configuration and monitoring

The post Claude Code: The best terminal AI Agent for software engineers appeared first on Cloudwork.

]]>
Advanced configuration and monitoring on AWS RDS /aws-rds-advanced-configuration-and-monitoring/ Mon, 17 Mar 2025 10:27:34 +0000 /?p=412 AWS RDS – Advanced configuration and monitoring There are plenty of guides available that demonstrate how to set up an RDS instance, so we won’t delve into that here.The primary aim of this article is to clarify how to configure logging and enable certain features on an RDS instance. These features are typically enabled by default for the root user on an on-premise MySQL database but require additional steps on RDS. Why RDS Doesn’t Let You Use Certain Features by Default By default, RDS does not grant superuser privileges (GRANT ALL) to any of the users you create — not even the admin user.So, if you try to create a MySQL function like this: CREATE FUNCTION `TESTFUNCTION`(s TEXT, defaultValue TEXT) RETURNS TEXT CHARSET latin1 DETERMINISTIC RETURN IF(s IS NULL OR s = ‘’, defaultValue, s); RDS will block it. This is a security feature. If an unauthorized user gains access, they won’t be able to tamper with sensitive functions. How to Enable Function Creation in RDS To enable administrative tasks (like creating a function), follow these steps: Go to the RDS > Parameter Groups section. Create a new parameter group. Set log_bin_trust_function_creators = 1. Associate this group with your RDS instance via the Edit section. ⚠️ IMPORTANT: Make sure the parameter group family matches the MySQL version of your RDS instance. Otherwise, you won’t be able to assign it. Once applied, you’ll be able to create the function. Enabling Detailed Logging in RDS Just checking the logging options during instance creation won’t fully enable logs.It only enables the capability to publish logs — not the logs themselves. By default, only error logs are active.   How to Enable Audit Logging To activate audit logging: Create a new Option Group. The default one cannot be edited. Go to Database Options, choose your new group. Add the MARIADB_AUDIT_PLUGIN with default parameters. You can customize things like file rotation later. Click Add Option. Assign the group to your instance. Once it’s active, you’ll find an audit.log file under the Logs & Events tab of your instance.   Enabling Slow and General Query Logs Create or edit a custom Parameter Group. Set: slow_query_log = 1 long_query_time = 5 log_output = file This setup logs queries that take over 5 seconds and stores them in a separate file — which helps avoid performance issues.If you don’t do this, logs are stored inside the database, which you’d have to query to access. To enable general logging too, just set general_log = 1. ⚠️ IMPORTANT: A restart is required for changes in the parameter group to take effect. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —  Struggling with advanced RDS configuration? Let us handle the complexity and help your application run like a dream.Explore our services and reach out today — we’re here to make it easy! — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —  ARTICLE DEFINITIONS: – Parameter Group:A Parameter Group is a collection of database engine configuration settings and parameters that you can apply to one or more RDS database instances. It allows you to customize various aspects of your database, such as memory allocation, query behavior, and more, to meet your specific application requirements. – Option Group:An Option Group is a collection of database features and functionalities that you can enable or disable for your RDS database instance. It allows you to add capabilities like encryption, automated backups, and high availability to your database, tailoring it to your application’s requirements. – Audit Log:An Audit Log, also known as the Database Audit Log, is a record of actions and events that occur within your RDS database instance. It helps you track who accessed the database, what operations were performed, and when they occurred. This log is essential for security and compliance purposes. – Error Log:The Error Log is a file that captures information about errors and issues encountered by your RDS database. It can include details about database errors, crashes, and other anomalies. Reviewing the error log helps administrators diagnose and troubleshoot problems in the database. – Slow Query Log:The Slow Query Log is a record of database queries that take a longer time to execute than a specified threshold. It helps identify and optimize inefficient or resource-intensive database queries, improving the overall performance of your database.

The post Advanced configuration and monitoring on AWS RDS appeared first on Cloudwork.

]]>

AWS RDS - Advanced configuration and monitoring

There are plenty of guides available that demonstrate how to set up an RDS instance, so we won’t delve into that here.
The primary aim of this article is to clarify how to configure logging and enable certain features on an RDS instance. These features are typically enabled by default for the root user on an on-premise MySQL database but require additional steps on RDS.

Why RDS Doesn’t Let You Use Certain Features by Default

By default, RDS does not grant superuser privileges (GRANT ALL) to any of the users you create — not even the admin user.
So, if you try to create a MySQL function like this:

CREATE FUNCTION `TESTFUNCTION`(s TEXT, defaultValue TEXT) RETURNS TEXT CHARSET latin1 DETERMINISTIC RETURN IF(s IS NULL OR s = ‘’, defaultValue, s);
RDS will block it. This is a security feature. If an unauthorized user gains access, they won’t be able to tamper with sensitive functions.

How to Enable Function Creation in RDS

To enable administrative tasks (like creating a function), follow these steps:

  1. Go to the RDS > Parameter Groups section.

  2. Create a new parameter group.

  3. Set log_bin_trust_function_creators = 1.

  4. Associate this group with your RDS instance via the Edit section.

⚠ IMPORTANT: Make sure the parameter group family matches the MySQL version of your RDS instance. Otherwise, you won’t be able to assign it.

Once applied, you’ll be able to create the function.

Enabling Detailed Logging in RDS

Just checking the logging options during instance creation won’t fully enable logs.
It only enables the capability to publish logs — not the logs themselves. By default, only error logs are active.

 

How to Enable Audit Logging

To activate audit logging:

  1. Create a new Option Group. The default one cannot be edited.

  2. Go to Database Options, choose your new group.

  3. Add the MARIADB_AUDIT_PLUGIN with default parameters. You can customize things like file rotation later.

  4. Click Add Option.

  5. Assign the group to your instance.

Once it’s active, you’ll find an audit.log file under the Logs & Events tab of your instance.

 

Enabling Slow and General Query Logs

  1. Create or edit a custom Parameter Group.

  2. Set:

    • slow_query_log = 1

    • long_query_time = 5

    • log_output = file

This setup logs queries that take over 5 seconds and stores them in a separate file — which helps avoid performance issues.
If you don’t do this, logs are stored inside the database, which you’d have to query to access.

To enable general logging too, just set general_log = 1.

⚠ IMPORTANT: A restart is required for changes in the parameter group to take effect.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 

Struggling with advanced RDS configuration? Let us handle the complexity and help your application run like a dream.
Explore our services and reach out today — we’re here to make it easy!

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 

ARTICLE DEFINITIONS:

– Parameter Group:
A Parameter Group is a collection of database engine configuration settings and parameters that you can apply to one or more RDS database instances. It allows you to customize various aspects of your database, such as memory allocation, query behavior, and more, to meet your specific application requirements.

– Option Group:
An Option Group is a collection of database features and functionalities that you can enable or disable for your RDS database instance. It allows you to add capabilities like encryption, automated backups, and high availability to your database, tailoring it to your application’s requirements.

– Audit Log:
An Audit Log, also known as the Database Audit Log, is a record of actions and events that occur within your RDS database instance. It helps you track who accessed the database, what operations were performed, and when they occurred. This log is essential for security and compliance purposes.

– Error Log:
The Error Log is a file that captures information about errors and issues encountered by your RDS database. It can include details about database errors, crashes, and other anomalies. Reviewing the error log helps administrators diagnose and troubleshoot problems in the database.

– Slow Query Log:
The Slow Query Log is a record of database queries that take a longer time to execute than a specified threshold. It helps identify and optimize inefficient or resource-intensive database queries, improving the overall performance of your database.

The post Advanced configuration and monitoring on AWS RDS appeared first on Cloudwork.

]]>