Skip to content

CLI Reference

Complete documentation for all Blacklake CLI commands.

Overview

Blacklake provides a comprehensive command-line interface for managing data repositories with semantic metadata. All commands follow git-like conventions for familiarity.

Command Structure

blacklake [GLOBAL_OPTIONS] COMMAND [COMMAND_OPTIONS] [ARGUMENTS]

Global Options

Option Description
--version Show version and exit
--help Show help and exit
--verbose Enable verbose output
--quiet Suppress output except errors

Available Commands

Usage:  [OPTIONS] COMMAND [ARGS]...

  Blacklake: A git-like data storage and retrieval system for S3/object
  stores.

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  add             Add file(s) to the staging area.
  branch          List, create, or delete branches.
  checkout        Checkout a branch or commit.
  commit          Commit staged changes.
  index           Index data into Solr.
  init            Initialize a new blacklake repository.
  log             Show commit logs.
  meta            Metadata management commands.
  pull            Pull commits from remote object store.
  push            Push commits to remote object store.
  remote-add      Add a remote repository.
  remote-set-url  Set remote repository URL.
  rm              Remove file(s) from the staging area.
  search          Search for data using Solr.
  status          Show the working tree status.
  tag             Create a tag.

Command Details

Repository Management

blacklake init

Initialize a new Blacklake repository.

blacklake init [OPTIONS]

Options: - --bare - Create bare repository - --s3-endpoint - S3 endpoint URL - --s3-bucket - S3 bucket name - --s3-region - S3 region (default: us-east-1) - --s3-access-key - S3 access key - --s3-secret-key - S3 secret key

Examples: 

# Initialize with MinIO
blacklake init \
  --s3-endpoint http://localhost:9000 \
  --s3-bucket my-data \
  --s3-access-key minioadmin \
  --s3-secret-key minioadmin

# Initialize with AWS S3
blacklake init \
  --s3-endpoint https://s3.amazonaws.com \
  --s3-bucket company-data-lake \
  --s3-region us-west-2

Expected Output: 

✓ Initialized empty blacklake repository in .bl/

blacklake status

Show working tree status.

blacklake status [OPTIONS]

Options: - --porcelain - Machine-readable output - --short - Short format output

Examples: 

# Show status
blacklake status

# Machine-readable format
blacklake status --porcelain

Expected Output: 

On branch main
Changes to be committed:
  data.csv (new file)

Untracked files:
  README.md

File Operations

blacklake add

Add files to the staging area.

blacklake add [OPTIONS] PATH...

Options: - --all - Add all files - --force - Force add ignored files

Examples: 

# Add specific file
blacklake add data.csv

# Add all files
blacklake add --all

# Add multiple files
blacklake add data.csv metadata.json

Expected Output: 

✓ Added data.csv to staging area

blacklake rm

Remove files from working tree and staging area.

blacklake rm [OPTIONS] PATH...

Options: - --cached - Remove from staging area only - --force - Force removal

Examples: 

# Remove file
blacklake rm data.csv

# Remove from staging only
blacklake rm --cached data.csv

Expected Output: 

✓ Removed data.csv

blacklake commit

Record changes to the repository.

blacklake commit [OPTIONS] -m MESSAGE

Options: - -m, --message - Commit message (required) - --author - Override author - --email - Override email

Examples: 

# Simple commit
blacklake commit -m "Add sales data"

# Commit with author
blacklake commit -m "Add sales data" --author "Data Team" --email "data@company.com"

Expected Output: 

✓ Changes committed

Metadata Commands

blacklake meta

Metadata management commands.

blacklake meta [COMMAND] [OPTIONS]

blacklake meta init

Initialize metadata system.

blacklake meta init

Expected Output: 

✓ Metadata system initialized

blacklake meta add

Add metadata to an object.

blacklake meta add [OPTIONS] PATH

Options: - --title - Object title - --description - Object description - --creator - Creator name - --email - Creator email - --keywords - Comma-separated keywords - --rights - Rights statement - --license - License URL

Examples: 

# Add metadata interactively
blacklake meta add data.csv --title "Sales Data"

# Add full metadata
blacklake meta add data.csv \
  --title "Sales Data 2024" \
  --description "Q1-Q4 sales data" \
  --creator "Data Team" \
  --email "data@company.com" \
  --keywords "sales,revenue,2024" \
  --rights "CC BY 4.0" \
  --license "https://creativecommons.org/licenses/by/4.0/"

Expected Output: 

✓ Metadata added for data.csv

blacklake meta validate

Validate metadata using SHACL shapes.

blacklake meta validate [OPTIONS] [FILE]

Options: - --all - Validate all metadata - --format - Output format (json, text)

Examples: 

# Validate specific file
blacklake meta validate data.csv.meta.jsonld

# Validate all metadata
blacklake meta validate --all

Expected Output: 

✓ Metadata is valid

blacklake meta export

Export metadata in specified format.

blacklake meta export [OPTIONS]

Options: - --format - Export format (ttl, nquads, jsonld)

Examples: 

# Export as Turtle
blacklake meta export --format ttl

# Export as N-Quads
blacklake meta export --format nquads

Expected Output: 

@prefix bl: <https://sempervent.github.io/blacklake/ns#> .
@prefix dct: <http://purl.org/dc/terms/> .

bl:obj:sha256:abc123... a dct:Dataset ;
  dct:title "Sales Data 2024" ;
  dct:creator [ a foaf:Person ;
    foaf:name "Data Team" ;
    foaf:mbox <mailto:data@company.com> ] .

Exit Codes

Code Description
0 Success
1 General error
2 Configuration error
3 Network error
4 Validation error

Examples

Complete Workflow

# Initialize repository
blacklake init --s3-endpoint http://localhost:9000 --s3-bucket my-data

# Add metadata to file
blacklake meta add data.csv --title "Sales Data" --creator "Data Team"

# Stage and commit
blacklake add data.csv
blacklake commit -m "Add sales data with metadata"

# Push to remote
blacklake push origin main

# Search data
blacklake search "sales"

Batch Operations

# Add multiple files with metadata
for file in *.csv; do
  blacklake meta add "$file" --title "$(basename "$file" .csv)" --creator "Data Team"
  blacklake add "$file"
done

blacklake commit -m "Add multiple datasets"
blacklake push origin main

Next Steps