File Storage

Overview

Spiderly's file-storage model is per-property: every blob property on an entity declares its storage adapter via a StorageAttribute subclass. The source generator emits the upload pipeline (controller endpoint, validation, optimization hooks, save-time cleanup) and resolves the right IFileManager adapter from DI for that property only — there is no global storage registration.

Built-in Adapters

Spiderly ships three built-in storage classes and matching attributes. Anything else (Cloudinary, Azure Blob, Backblaze, on-prem MinIO, …) is implemented by the consumer (see Custom Adapters).

All three implement Spiderly.Shared.Interfaces.IFileManager.

Entity Configuration

Decorate a string property with a storage attribute. The attribute itself marks the property as a blob — there is no separate marker attribute.

Public CDN file

The column stores the full public URL. Ideal for images served directly from a CDN:

public class Brand : BusinessObject<int>
{
    [S3PublicStorage]
    [AcceptedFileTypes("image/jpeg", "image/png", "image/webp", "image/avif")]
    [MaxFileSize(2_000_000)]
    [StringLength(1000, MinimumLength = 1)]
    public string LogoUrl { get; set; }
}

Private S3 file (signed-URL access)

The column stores an opaque S3 key; access is mediated via signed URLs or a backend proxy. Use for personal data, compliance-sensitive uploads, etc.:

public class WarrantyRegistration : BusinessObject<long>
{
    [S3PrivateStorage]
    [AcceptedFileTypes("image/jpeg", "image/png", "application/pdf")]
    [MaxFileSize(10_000_000)]
    [StringLength(1000, MinimumLength = 1)]
    public string ReceiptImageUrl { get; set; }
}

Local disk (development)

public class User : BusinessObject<long>
{
    [DiskStorage]
    [AcceptedFileTypes("image/*")]
    [StringLength(1000, MinimumLength = 1)]
    public string ProfilePicture { get; set; }
}

File Validation Attributes

These attributes add both server-side and client-side validation. See the Validation page for details.

Example with all validation attributes

public class Brand : BusinessObject<int>
{
    [DisplayName]
    [Required]
    [StringLength(100, MinimumLength = 1)]
    public string Name { get; set; }

    [S3PublicStorage]
    [AcceptedFileTypes("image/*")]
    [MaxFileSize(2_000_000)]
    [ImageWidth(400)]
    [ImageHeight(400)]
    [StringLength(1000, MinimumLength = 1)]
    public string Logo { get; set; }
}

Provider Setup

S3 (public or private)

Both S3PublicStorageService and S3PrivateStorageService share a single IAmazonS3 registration.

appsettings.json:

{
  "AppSettings": {
    "Spiderly.Shared": {
      "S3BucketName": "my-bucket",
      "S3PublicEndpoint": "https://cdn.example.com"
    }
  }
}

S3PublicEndpoint is the base URL S3PublicStorageService uses to format returned URLs as {S3PublicEndpoint}/{key}. The setting is unused by S3PrivateStorageService. S3 credentials (S3AccessKey, S3SecretKey, S3ServiceUrl) live in your application's settings, not in Spiderly.Shared.

DI registration (your AppServiceExtensions or equivalent):

services.AddSingleton<IAmazonS3>(sp =>
{
    IConfiguration configuration = sp.GetRequiredService<IConfiguration>();
    AmazonS3Config s3Config = new AmazonS3Config
    {
        ServiceURL = configuration.GetValue<string>($"{Spiderly.Shared.Settings.ConfigurationSection}:S3ServiceUrl"),
        ForcePathStyle = true,
        AuthenticationRegion = "auto",
    };

    return new AmazonS3Client(
        new BasicAWSCredentials(
            configuration.GetValue<string>($"{Spiderly.Shared.Settings.ConfigurationSection}:S3AccessKey"),
            configuration.GetValue<string>($"{Spiderly.Shared.Settings.ConfigurationSection}:S3SecretKey")
        ),
        s3Config
    );
});

services.AddTransient<S3PublicStorageService>();
services.AddTransient<S3PrivateStorageService>();

The source generator emits _deps.ServiceProvider.GetRequiredService<TConcrete>() per blob property, so each adapter you reference must be registered by its concrete type.

Disk (local development)

No appsettings entries needed. Files are stored under {CurrentDirectory}/FileStorage.

services.AddTransient<DiskStorageService>();

DiskStorageService is intended for development. In a Linux/Docker production deployment the host filesystem is ephemeral and not shared across replicas; switch to S3PublicStorage / S3PrivateStorage (or a custom adapter) for production.

Custom Adapters

Spiderly ships only the three adapters above. Other backends (Cloudinary, Azure Blob, Backblaze B2, on-prem MinIO, …) are written by the consumer.

Step 1 — Implement IFileManager

public class CloudinaryStorageService : IFileManager
{
    public Task<string> UploadFileAsync(...)             { /* impl */ }
    public Task DeleteNonActiveBlobs(...)                { /* impl */ }
    public Task<string> GetFileDataAsync(string key)     { /* impl */ }
    public Task<string> MoveBlobToEntityPathAsync(...)   { /* impl */ }
    public Task DeleteNonActiveEditorImages(...)         { /* impl */ }
}

Step 2 — Subclass StorageAttribute

Pass your service type to the base constructor:

public sealed class CloudinaryStorageAttribute : StorageAttribute
{
    public CloudinaryStorageAttribute() : base(typeof(CloudinaryStorageService)) { }
}

Step 3 — Register the service

services.AddTransient<CloudinaryStorageService>();

Step 4 — Use the attribute on entity properties

public class User : BusinessObject<long>
{
    [CloudinaryStorage]
    [AcceptedFileTypes("image/*")]
    [StringLength(500, MinimumLength = 1)]
    public string Photo { get; set; }
}

The source generator detects custom storage attributes by the convention "attribute simple name ends with Storage" and treats the property as a blob automatically. Field-name resolution in the generator is currently hard-coded for the three built-ins; for custom adapters in auto-generated CRUD, you may need to inject your custom service directly into hand-written upload paths until Spiderly's source generator gains symbol-level resolution of StorageAttribute.ServiceType.

Generated Upload Pipeline

Upload flow

1. Client sends POST /api/{Entity}/Upload{Property}For{Entity} with the file
2. OnBefore{Property}BlobFor{Entity}UploadIsAuthorized() hook runs
3. Authorization check (insert vs update based on entity ID)
4. File size validation — [MaxFileSize] if set, otherwise 20 MB default
5. MIME-type + magic-byte signature validation — [AcceptedFileTypes] is required on every blob property and must declare at least one MIME-typed value (e.g. [AcceptedFileTypes("image/jpeg", "image/png", "image/webp", "image/avif")]). If it is missing or contains only extension values, the source generator emits build error SPIDERLY014. The server reads the first 16 bytes of the stream and rejects requests whose content does not match the declared Content-Type — spoofing the header does not bypass validation.
6. OnBefore{Property}BlobFor{Entity}IsUploaded() hook runs — for images, this validates dimensions and optimizes
7. File is uploaded to the storage adapter resolved per the property's [*Storage] attribute
8. The file identifier (key or URL) is returned to the client

Rate Limiting

All generated Upload*For* endpoints are decorated with [EnableRateLimiting(SpiderlyRateLimitPolicies.BlobUpload)]. Calling spiderly.AddRateLimiting() in your AddSpiderly(...) setup registers the policy with a default of 20 requests per minute per IP. Override the policy in your own Configure<RateLimiterOptions> call to tune the limit without forking Spiderly.

Default image processing

For image files, the default OnBefore{Property}BlobFor{Entity}IsUploaded hook:

1. Validates dimensions — if [ImageWidth] or [ImageHeight] are set, checks exact pixel dimensions
2. Optimizes — converts to WebP format at 85% quality using SixLabors.ImageSharp

File Processing Hooks

All hooks are virtual methods on the generated entity service class (e.g., ProductServiceGenerated). Override them in your entity service class (e.g., ProductService) to customize behavior.

Example: custom image optimization

Override the optimization hook to resize images before storage:

public override async Task<byte[]> OptimizeImageForLogoOfBrand(
    Stream stream, IFormFile file, int id)
{
    return await Helper.OptimizeImage(
        stream,
        newImageSize: new Size(400, 400),
        quality: 90
    );
}

Example: skip optimization for a specific property

public override async Task<byte[]> OptimizeImageForBannerOfHomePage(
    Stream stream, IFormFile file, long id)
{
    return await Helper.ReadAllBytesAsync(stream);
}

Displaying Files

How uploaded files appear in DTOs depends on the storage adapter.

DTO generation

For every blob property, Spiderly generates a companion {Property}Data field on the DTO:

// Entity:
public string ProfilePicture { get; set; }

// Generated DTO:
public string ProfilePicture { get; set; }     // storage key or URL
public string ProfilePictureData { get; set; }  // file content for display

What {Property}Data contains

In the Angular admin panel, spiderly-file handles this automatically. It uses the [isUrlFileData] input (auto-generated) to determine how to render the preview.

Storage Paths and Orphan Cleanup

All adapters place uploaded blobs under a hierarchical, entity-scoped key:

{EntityName}/{PropertyName}/{ObjectId}/{BlobGuid}.{ext}

Insert flow — staging prefix

When a user uploads a file for an entity that doesn't exist yet (insert), the entity ID is 0. Spiderly routes these uploads to a temporary staging prefix:

{EntityName}/{PropertyName}/_tmp/{UploadGuid}/{BlobGuid}.{ext}

Once the entity is saved and has a real ID, the generated save code calls IFileManager.MoveBlobToEntityPathAsync(...), which copies the blob to its permanent key ({Entity}/{Prop}/{realId}/{BlobGuid}.ext), deletes the staging source, and updates the DB column. The client never sees the staged path.

Update flow — replace and clean

When a user replaces a file on an existing entity:

1. User uploads a new file → new key/URL is returned
2. User saves the entity with the new key/URL
3. After SaveChangesAsync(), the generated code calls DeleteNonActiveBlobs() on the storage adapter
4. The adapter lists all files under {Entity}/{Prop}/{id}/ and deletes everything except the active file

This design is intentional — files are uploaded before the entity is saved (so the upload endpoint works independently). Cleanup only happens at save time, which means refreshing the page without saving won't lose the old file.