Memory-FS is a fully tested, type-safe file system abstraction implemented entirely in memory. It originated from the OSBot_Cloud_FS
prototype and has evolved through multiple architectural iterations documented in docs/tech_debriefs. The library exposes a small, composable API for creating, loading and editing file objects while remaining agnostic of the underlying storage provider.
- In-Memory Storage – extremely fast read/write operations via an in‑memory provider.
- Type Safety – all APIs use
osbot_utils
Type_Safe
models ensuring runtime validation. - Two-File Pattern – metadata (
.config
/.metadata
) and content are stored separately. - Pluggable Path Strategies – configurable path handlers (latest, temporal, versioned, etc.).
- Extensible File Types – JSON, Markdown, binary and custom formats through schema classes.
- 100% Code Coverage – every line of production code is tested as of version
v0.11.1
.
The current design (introduced in the June 18 2025 debrief) is composed of two layers:
- File_FS Layer – high level file operations
File_FS
object plusFile_FS__*
action classesTarget_FS
factory for creating file objects from existing paths
- Storage_FS Layer – low level storage providers
Storage_FS
interface and in‑memory implementationStorage_FS__Memory
- Provider pattern enabling future backends (local disk, SQLite, zip, S3)
This separation yields a small surface area for extending or replacing storage while keeping file logic consistent. The project’s tech debrief README describes this architecture as:
1. **File_FS Layer**: High-level file operations
2. **Storage_FS Layer**: Low-level storage operations
- Provider pattern for pluggable storage backends
Files are organised using a predictable naming scheme:
- Content File –
{file_id}.{extension}
- Config File –
{file_id}.{extension}.config
- Metadata File –
{file_id}.{extension}.metadata
Memory-FS maintains comprehensive tests. As recorded in the debriefs:
As of June 18, 2025 (v0.11.0), Memory-FS has achieved **100% code coverage**:
- Every line of production code is tested
- All edge cases are covered
- No dead or unused code remains
- Comprehensive test suite with 200+ test methods
pip install memory-fs
or install from source:
pip install -e .
Below is a minimal example using the high‑level API. It stores JSON content via the in‑memory provider:
from memory_fs.file_fs.File_FS import File_FS
from memory_fs.file_types.Memory_FS__File__Type__Json import Memory_FS__File__Type__Json
from memory_fs.schemas.Schema__Memory_FS__File__Config import Schema__Memory_FS__File__Config
from memory_fs.path_handlers.Path__Handler__Latest import Path__Handler__Latest
from memory_fs.storage.Memory_FS__Storage import Memory_FS__Storage
from memory_fs.storage_fs.providers.Storage_FS__Memory import Storage_FS__Memory
# set up in-memory storage
storage = Memory_FS__Storage(storage_fs=Storage_FS__Memory())
# configuration describing where/how the file should be stored
config = Schema__Memory_FS__File__Config(
path_handlers = [Path__Handler__Latest()],
default_handler=Path__Handler__Latest,
file_type = Memory_FS__File__Type__Json(),
)
# create a file object and save some data
file = File_FS(file_config=config, storage=storage)
file.create()
file.create__content(b'{"hello": "world"}')
# reload using the factory
from memory_fs.target_fs.Target_FS__Create import Target_FS__Create
loaded_target = Target_FS__Create(storage=storage).from_path__config(file.file_fs__paths().paths__config()[0])
print(loaded_target.file_fs().content()) # -> b'{"hello": "world"}'
Extensive documentation is kept under the docs
folder. The tech_debriefs directory captures the project evolution, while architecture provides a detailed design overview. Developers should also review docs/dev for coding conventions, bug lists and TODO items.
The debrief timeline in docs/tech_debriefs/README.md
summarises all key milestones from the initial design on May 26 2025 through the storage abstraction layer, Target_FS introduction and the 100% coverage refactor. The current release is v0.11.1
.
Memory-FS serves both as a lightweight in‑memory storage engine and as a reference implementation for future backends such as S3 or SQLite. Its small, type‑safe API and complete test coverage make it an ideal starting point for new contributors or for embedding a fast, pluggable file system into your own projects.