Data Organization

Data Organization

This directory contains data files for blog posts, organized by date.

Structure

assets/data/
├── 2026-02-23/          # Decomposer-associated fungi (ITS amplicon sequencing)
│   ├── *.csv            # Analysis outputs (alpha diversity, PCoA, etc.)
│   ├── *.tsv            # Metadata and feature tables
│   └── exported-quality-data/
└── 2026-02-24/          # Fly larvae CO1 barcoding (Nanopore sequencing)
    ├── barcode_read_counts.csv
    ├── consensus_build_summary.csv
    ├── species_id_final_annotated.csv
    └── throughput_*.csv

Adding New Data

When creating a new blog post:

1. Create a new dated directory: assets/data/YYYY-MM-DD/
2. Place all data files for that post in the directory
3. Create corresponding JavaScript file: assets/js/YYYY-MM-DD-plots.js
4. Update the JavaScript to load data from /assets/data/YYYY-MM-DD/
5. Create blog post: _posts/YYYY-MM-DD-title.md

Data File Naming Conventions

• Use lowercase with underscores: alpha_diversity.csv
• Be descriptive: species_id_final_annotated.csv not results.csv
• Include analysis type or stage in name
• Use standard extensions: .csv for comma-separated, .tsv for tab-separated

Blog Post Template

---
layout: post
title: "Your Title"
date: YYYY-MM-DD
categories: [category1, category2]
---

## Introduction
[Background and context]

## Methods
[Experimental design and analysis pipeline]

## Results
[Figures and interpretations]

## Discussion
[Interpretation and conclusions]

---

<script src="https://cdn.jsdelivr.net/npm/d3@7"></script>
<script src="https://cdn.jsdelivr.net/npm/@observablehq/plot@0.6"></script>
<script src="/assets/js/YYYY-MM-DD-plots.js"></script>

JavaScript Template

// Observable Plot visualizations for YYYY-MM-DD post
const { Plot, d3 } = window;

// Helper to load CSV
async function loadCSV(filename) {
  const response = await fetch(`/assets/data/YYYY-MM-DD/${filename}`);
  const text = await response.text();
  return d3.csvParse(text, d3.autoType);
}

// Plot functions...
async function plotExample() {
  const data = await loadCSV('data_file.csv');
  // Create plot...
}

// Initialize
(async function() {
  try {
    await plotExample();
    console.log('All plots rendered successfully');
  } catch (error) {
    console.error('Error rendering plots:', error);
  }
})();

Notes

• Data files are tracked in git for reproducibility
• Large files (>100MB) should be compressed or stored externally
• Include raw data when possible, not just processed results
• Document data processing scripts in the corresponding dated directory