Filestore, Memorystore and Moving Data

GCP Basics · Storage & Databases – Cloud Storage, Cloud SQL and Choosing a Database · Lesson 5 of 5

Filestore gives VMs and GKE a managed NFS share, and Storage Transfer Service, Transfer Appliance and gcloud storage move data from other clouds and data centers into Google Cloud.

Not everything is an object or a database row. Many apps need a shared file system, and every migration needs a way to move data in.

flowchart LR
  VMS["VMs and GKE Pods"] -- "NFS mount" --> FS["Filestore share /shared"]
  S3["AWS S3 bucket"] -- "scheduled job" --> STS["Storage Transfer Service"]
  ONP["On-premises NAS"] -- "transfer agents" --> STS
  STS --> GCS["Cloud Storage bucket"]
  TA["Transfer Appliance<br/>hundreds of TB"] -- "shipped" --> GCS
  %% step VMS: Several VMs or Pods mount the same share for content, home folders or legacy apps.
  %% step FS: Filestore is a managed NFS server with predictable performance; no NFS VM to run.
  %% step S3: Data in another cloud is copied on a schedule with no servers of your own.
  %% step ONP: For data centers, transfer agents run near the data and stream it to Google Cloud.
  %% step STS: Storage Transfer Service checks integrity, retries failures and can delete or keep the source.
  %% step TA: For very large datasets or slow links, Google ships a storage device that you fill and send back.

Filestore

Tier	Notes
Basic HDD / Basic SSD	Simple shares from 1 TiB; zonal
Zonal	Higher performance, scales capacity
Regional	Replicated across zones for high availability

Clients use NFSv3 (NFSv4.1 on some tiers); mount with mount -t nfs 10.10.3.2:/shared /mnt/shared.
In GKE, the Filestore CSI driver provides ReadWriteMany volumes.
Backups and snapshots protect shares.
NetApp Volumes offers NFS and SMB with NetApp features; Managed Lustre serves HPC and AI training.

Memorystore reminder

Memorystore (Valkey, Redis Cluster, Redis, Memcached) is a managed cache. Use the Standard tier (with a replica) for anything that must survive a zone failure, and remember it is not durable storage.

Moving data into Google Cloud

Situation	Tool
A few GB from a laptop or VM	`gcloud storage cp` / `rsync`
Large or scheduled copies from S3, Azure Blob, HTTP lists, other buckets	Storage Transfer Service (managed, no servers)
On-premises file systems over the network	Storage Transfer Service with agents
Hundreds of TB with limited bandwidth	Transfer Appliance
Databases	Database Migration Service (MySQL, PostgreSQL, SQL Server, Oracle to Cloud SQL/AlloyDB)

A rough rule: 100 TB over a 1 Gbps link takes about 12 days at full speed. When that is too slow, ship an appliance.

gcloud filestore instances create shared-fs --zone=asia-southeast1-a --tier=BASIC_HDD   --file-share=name=shared,capacity=1TB --network=name=vpc-shop
gcloud transfer jobs create s3://legacy-reports gs://dl-reports   --source-creds-file=aws-creds.json --schedule-repeats-every=1d

Command	Meaning
`gcloud filestore instances list`	Shares and their IPs
`gcloud transfer jobs list`	Transfer jobs
`gcloud transfer operations list --job-names=…`	Runs of a job

Traps: Filestore capacity has a minimum size per tier, so tiny shares cost more than you expect; Memorystore Basic tier has no replica; Transfer Service needs read access on the source and write access for its service agent on the destination bucket.