Skip to content

I/O functions

Functions that read and write metacooler files. All of the entry points below are re-exported at the top level, so metacooler.from_pairs(...) and metacooler.io.create.from_pairs(...) are equivalent:

Top-level name Defined in
from_pairs, from_cool metacooler.io.create
collect metacooler.io.collect
reassign metacooler.io.reassign
annotate metacooler.io.annotate
extract metacooler.io.extract
flatten metacooler.io.flatten
merge metacooler.io.merge
network metacooler.io.network
dump_table metacooler.io.dump
create_mmetacool, add_entry, remove_entry, copy_entry, copy_entry_cross_file, move_entry, list_entries metacooler.io.mmetacool
is_metacooler, is_mmetacooler, validate, validate_mmetacooler metacooler.schema
parse_uri metacooler.uri

extract, flatten, merge, network, dump_table, validate, and validate_mmetacooler were previously reachable only through their submodules.

Note that network and dump_table name their output parameter out, not output.

A failed write

from_pairs, from_cool, collect, reassign and annotate (the last two without inplace) build their output as <output>.partial and move it into place only once it has validated, so a failed run never leaves a half-written file at the name asked for, nor touches a file already there. from_pairs stages its .ps.tsv table the same way and moves it into place just before the map, the previous table set aside (as <name>.ps.tsv.previous) until the map is in place and put back if the map cannot be moved. Once the map is in place the set-aside table is removed; if that fails, it is left where it is, with a warning, and not put back: the new map and table match, and the run succeeds unless it was interrupted. Only a process killed during those moves can leave them half done: the previous table still set aside, or the new table beside the previous map. Either way, from_pairs refuses to start while a <name>.ps.tsv.previous exists, before it reads anything: check which table matches the map, keep it as <name>.ps.tsv, and remove the other (a file of your own of that name is refused alike: rename it). reassign measures the QC block before it writes anything. What becomes of <output>.partial is keep_partial's choice:

keep_partial When the write fails
True (default) The partial file stays, for inspection, and an ERROR line says where. Interrupted with Ctrl-C (KeyboardInterrupt), it stays without a word.
False The partial file is removed whatever stopped the run, KeyboardInterrupt included, and an INFO line says so. For a caller that stages its own output and wants the finished file or nothing.

A run that fails before its partial file exists has nothing to keep or remove, and says nothing about one.

import metacooler

try:
    metacooler.reassign(
        "sample.metacool", "bins.tsv", "sample.binned.metacool", keep_partial=False
    )
except Exception:
    ...  # neither sample.binned.metacool nor its .partial exists

Create metacool files

Create metacool files from pairs, bins, and MAG assignments.

from_pairs

from_pairs(
    uri: str | Path,
    bins: str | Path | DataFrame | None = None,
    pairs: str | Path | Sequence[str | Path] | None = None,
    contig_genome_map: dict[str, str] | str | Path | None = None,
    contig_table_format: str = CONTIGS_FORMAT_DEFAULT,
    fasta: str | Path | None = None,
    assembly_name: str | None = None,
    contig_ordering: Literal[
        "genome-then-length", "genome-then-alpha", "fasta", "alpha"
    ] = "genome-then-length",
    drop_unassigned: bool = False,
    filters: Sequence[str] | None = None,
    binsize: int | None = None,
    genomes_table: str | Path | None = None,
    genomes_table_format: str = GENOMES_FORMAT_DEFAULT,
    pairs_kwargs: dict | None = None,
    metadata: dict | None = None,
    chunksize: int = DEFAULT_CHUNKSIZE,
    sort_in_memory: bool | None = None,
    qc: bool = True,
    qc_distance_threshold: int = DEFAULT_DISTANCE_THRESHOLD,
    ps: bool = True,
    ps_bins_per_decade: int = DEFAULT_BINS_PER_DECADE,
    force: bool = False,
    nproc: int = 1,
    keep_partial: bool = True,
) -> None

Create a .metacool file from pairs, bins, and contig assignments.

Parameters:

Name Type Description Default
uri str | Path

Output file path (.metacool extension recommended).

required
bins str | Path | DataFrame | None

Path to a BED file of bins (without header), or a pre-built DataFrame with columns chrom, start, end. If None, binsize must be given and bins will be tiled from the fasta.

None
pairs str | Path | Sequence[str | Path] | None

Path to a pairs file (plain text, gzipped, or bgzipped), or a sequence of them -- typically one per Hi-C library of the same assembly. Several files are read back to back in the given order, in one pass, as if they were one concatenated file: their contacts are summed, and the QC block, the P(s) table and the drop report cover them all, exactly -- where per-library maps joined by merge lose the QC block and have no P(s) table. Their #chromsize: headers are united, and one contig given two different lengths is refused, as is the same file given twice.

None
contig_genome_map dict[str, str] | str | Path | None

Either a dict[contig_name → genome_name] or a path to a contig table file (see contig_table_format). Contigs it does not list are collected under __unbinned__, and names that are not contigs of the assembly are ignored with a warning, but it must name at least one contig of the assembly. None builds a map with no binning yet: a cooler plus the genome layer, every contig in __unbinned__ (bin them later with reassign).

None
contig_table_format str

Which columns of a table contig_genome_map hold the contig and the genome names: CONTIG,GENOME[,noheader], each a 1-based position or a header name, or an alias (see :func:metacooler.io.contigs.parse_contigs_table).

CONTIGS_FORMAT_DEFAULT
fasta str | Path | None

Assembly fasta (required if bins is None).

None
assembly_name str | None

Name for this assembly / metagenome.

None
contig_ordering Literal['genome-then-length', 'genome-then-alpha', 'fasta', 'alpha']

Canonical ordering strategy.

'genome-then-length'
drop_unassigned bool

If True, drop contigs not present in the contig table instead of assigning them to __unbinned__.

False
filters Sequence[str] | None

--filter expressions (see :mod:metacooler.filters) applied to the derived /genomes and /contigs tables before anything is binned, e.g. ["contigs.length >= 5kb"]. A dropped contig gets no bins, and its pairs are discarded while binning. genomes.n_bins is not available here.

None
binsize int | None

Fixed bin size (used only when bins is None).

None
genomes_table str | Path | None

Optional path to a genomes metadata TSV. One column holds the genome names (see genomes_table_format); every other column is stored in the /genomes table.

None
genomes_table_format str

Which column of genomes_table holds the genome names: a 1-based position or a header name, plus ,noheader for a table without a header line. Defaults to genome_name.

GENOMES_FORMAT_DEFAULT
pairs_kwargs dict | None

Extra kwargs passed to the pairs reader (column indices, etc.), the same for every pairs file.

None
metadata dict | None

Additional metadata dict stored in file attrs.

None
chunksize int

Number of pairs to process per chunk. This is chunk granularity, not read memory: pyarrow's read block is capped separately at _PAIRS_READ_BLOCK_MAX_BYTES, and the binned batches are coalesced back up to chunksize afterwards. Under sort_in_memory=False every chunk costs one cooler.create call, whose runtime is dominated by a fixed overhead that scales with contig count rather than pixels (seconds per call on a metagenome), so small chunks are disproportionately expensive. The cost of a large value is the pixels held while coalescing one chunk — roughly chunksize rows at 24 bytes in the worst case.

DEFAULT_CHUNKSIZE
sort_in_memory bool | None

If True, collect all pixels in RAM and sort before writing (fast for small files). If False, pass the unsorted stream to cooler's external sort (scalable for large files). If None (default), buffer the binned pixels and use True unless they exceed 100 M rows, in which case fall back to False with a logged warning.

None
qc bool

Tally contact-quality statistics while the pairs are being binned and store them in metadata["qc"] and the qc_* columns of /genomes and /contigs (see :mod:metacooler.qc). Free apart from a few vectorised operations per chunk, since the pass over the pairs is happening anyway — and only possible here, because binning discards the sub-bin distances the short/long split needs. The per-contig short/long counts are what lets a later :func:metacooler.io.reassign.reassign keep that split.

True
qc_distance_threshold int

Intra-contig short/long split for qc, in bp.

DEFAULT_DISTANCE_THRESHOLD
ps bool

Accumulate a per-genome P(s) curve from the same pass and write it beside the output as <name>.ps.tsv (see :mod:metacooler.ps). Measured on read positions, so it resolves distances the binned map cannot — which is the whole reason to do it here rather than afterwards. Costs one searchsorted and one bincount per chunk over the intra-contig rows.

True
ps_bins_per_decade int

Log-grid resolution of the P(s) curve.

DEFAULT_BINS_PER_DECADE
force bool

Overwrite an existing <name>.ps.tsv. The .metacool itself is guarded by the caller (the CLI checks it before any work starts).

False
nproc int

Threads binning pairs batches while the file is being read and decompressed (which stays on one thread, being one gzip stream). Results are folded in file order, so the output and every tally are those of nproc=1; memory grows by one batch per thread.

1
keep_partial bool

On a failure while the output is being written, leave <uri>.partial behind for inspection, as the default does (logged at ERROR). False removes it instead, whatever stopped the run (KeyboardInterrupt included), and logs that at INFO. The output path itself is never written by a failed run, nor is the .ps.tsv table, staged as <name>.ps.tsv.partial and moved into place just before the map: the previous table is set aside (as <name>.ps.tsv.previous) until the map is in place, and put back if the map cannot be. Once the map is in place the set-aside table is removed; if that fails it is left, with a WARNING, and not put back: the new map and table match, and the run succeeds unless it was interrupted. Only a process killed during those moves can leave them half done: the previous table still set aside, or the new table beside the previous map.

True

Raises:

Type Description
ValueError

If nproc is below 1, no pairs file is given, the pairs files are not compatible (see pairs), or contig_genome_map names no contig of the assembly (checked before any pair is read).

FileNotFoundError

If a pairs file does not exist -- checked before any is read, so a typo in the last of several libraries does not cost a binning pass over the others.

FileExistsError

If <name>.ps.tsv.previous exists, with or without ps. A run leaves it when it was killed while it moved its map and table into place, or could not remove that table afterwards, and which table matches the map is for the user to check; a file of the user's own of that name is refused alike. Checked before anything is read.

Source code in src/metacooler/io/create.py
2776
2777
2778
2779
2780
2781
2782
2783
2784
2785
2786
2787
2788
2789
2790
2791
2792
2793
2794
2795
2796
2797
2798
2799
2800
2801
2802
2803
2804
2805
2806
2807
2808
2809
2810
2811
2812
2813
2814
2815
2816
2817
2818
2819
2820
2821
2822
2823
2824
2825
2826
2827
2828
2829
2830
2831
2832
2833
2834
2835
2836
2837
2838
2839
2840
2841
2842
2843
2844
2845
2846
2847
2848
2849
2850
2851
2852
2853
2854
2855
2856
2857
2858
2859
2860
2861
2862
2863
2864
2865
2866
2867
2868
2869
2870
2871
2872
2873
2874
2875
2876
2877
2878
2879
2880
2881
2882
2883
2884
2885
2886
2887
2888
2889
2890
2891
2892
2893
2894
2895
2896
2897
2898
2899
2900
2901
2902
2903
2904
2905
2906
2907
2908
2909
2910
2911
2912
2913
2914
2915
2916
2917
2918
2919
2920
2921
2922
2923
2924
2925
2926
2927
2928
2929
2930
2931
2932
2933
2934
2935
2936
2937
2938
2939
2940
2941
2942
2943
2944
2945
2946
2947
2948
2949
2950
2951
2952
2953
2954
2955
2956
2957
2958
2959
2960
2961
2962
2963
2964
2965
2966
2967
2968
2969
2970
2971
2972
2973
2974
2975
2976
2977
2978
2979
2980
2981
2982
2983
2984
2985
2986
2987
2988
2989
2990
2991
2992
2993
2994
2995
2996
2997
2998
2999
3000
3001
3002
3003
3004
3005
3006
3007
3008
3009
3010
3011
3012
3013
3014
3015
3016
3017
3018
3019
3020
3021
3022
3023
3024
3025
3026
3027
3028
3029
3030
3031
3032
3033
3034
3035
3036
3037
3038
3039
3040
3041
3042
3043
3044
3045
3046
3047
3048
3049
3050
3051
3052
3053
3054
3055
3056
3057
3058
3059
3060
3061
3062
3063
3064
3065
3066
3067
3068
3069
3070
3071
3072
3073
3074
3075
3076
3077
3078
3079
3080
3081
3082
3083
3084
3085
3086
3087
3088
3089
3090
3091
3092
3093
3094
3095
3096
3097
3098
3099
3100
3101
3102
3103
3104
3105
3106
3107
3108
3109
3110
3111
3112
3113
3114
3115
3116
3117
3118
3119
3120
3121
3122
3123
3124
3125
3126
3127
3128
3129
3130
3131
3132
3133
3134
3135
3136
3137
3138
3139
3140
3141
3142
3143
3144
3145
3146
3147
3148
3149
3150
3151
3152
3153
3154
3155
3156
3157
3158
3159
3160
3161
3162
3163
3164
3165
3166
3167
3168
3169
3170
3171
3172
3173
3174
3175
3176
3177
3178
3179
3180
3181
3182
3183
3184
3185
3186
3187
3188
3189
3190
3191
3192
3193
3194
3195
3196
3197
3198
3199
3200
3201
3202
3203
3204
3205
3206
3207
3208
3209
3210
3211
3212
3213
3214
3215
3216
3217
3218
3219
3220
3221
3222
3223
3224
3225
3226
3227
3228
3229
3230
3231
3232
3233
def from_pairs(
    uri: str | Path,
    bins: str | Path | pd.DataFrame | None = None,
    pairs: str | Path | Sequence[str | Path] | None = None,
    contig_genome_map: dict[str, str] | str | Path | None = None,
    contig_table_format: str = CONTIGS_FORMAT_DEFAULT,
    fasta: str | Path | None = None,
    assembly_name: str | None = None,
    contig_ordering: Literal[
        "genome-then-length", "genome-then-alpha", "fasta", "alpha"
    ] = "genome-then-length",
    drop_unassigned: bool = False,
    filters: Sequence[str] | None = None,
    binsize: int | None = None,
    genomes_table: str | Path | None = None,
    genomes_table_format: str = GENOMES_FORMAT_DEFAULT,
    pairs_kwargs: dict | None = None,
    metadata: dict | None = None,
    chunksize: int = DEFAULT_CHUNKSIZE,
    sort_in_memory: bool | None = None,
    qc: bool = True,
    qc_distance_threshold: int = DEFAULT_DISTANCE_THRESHOLD,
    ps: bool = True,
    ps_bins_per_decade: int = DEFAULT_BINS_PER_DECADE,
    force: bool = False,
    nproc: int = 1,
    keep_partial: bool = True,
) -> None:
    """Create a ``.metacool`` file from pairs, bins, and contig assignments.

    Args:
        uri: Output file path (``.metacool`` extension recommended).
        bins: Path to a BED file of bins (without header), or a pre-built DataFrame
            with columns ``chrom, start, end``.  If ``None``, *binsize* must be given
            and bins will be tiled from the fasta.
        pairs: Path to a pairs file (plain text, gzipped, or bgzipped), or a
            sequence of them -- typically one per Hi-C library of the same
            assembly.  Several files are read back to back in the given order,
            in one pass, as if they were one concatenated file: their contacts
            are summed, and the QC block, the P(s) table and the drop report
            cover them all, exactly -- where per-library maps joined by
            ``merge`` lose the QC block and have no P(s) table.  Their
            ``#chromsize:`` headers are united, and one contig given two
            different lengths is refused, as is the same file given twice.
        contig_genome_map: Either a ``dict[contig_name → genome_name]`` or a path
            to a contig table file (see *contig_table_format*).  Contigs it
            does not list are collected under ``__unbinned__``, and names that
            are not contigs of the assembly are ignored with a warning, but it
            must name at least one contig of the assembly.  ``None`` builds a
            map with no binning yet: a cooler plus the genome layer, every
            contig in ``__unbinned__`` (bin them later with ``reassign``).
        contig_table_format: Which columns of a table *contig_genome_map* hold
            the contig and the genome names: ``CONTIG,GENOME[,noheader]``, each
            a 1-based position or a header name, or an alias (see
            :func:`metacooler.io.contigs.parse_contigs_table`).
        fasta: Assembly fasta (required if *bins* is ``None``).
        assembly_name: Name for this assembly / metagenome.
        contig_ordering: Canonical ordering strategy.
        drop_unassigned: If ``True``, drop contigs not present in the contig
            table instead of assigning them to ``__unbinned__``.
        filters: ``--filter`` expressions (see :mod:`metacooler.filters`)
            applied to the derived ``/genomes`` and ``/contigs`` tables before
            anything is binned, e.g. ``["contigs.length >= 5kb"]``.  A dropped
            contig gets no bins, and its pairs are discarded while binning.
            ``genomes.n_bins`` is not available here.
        binsize: Fixed bin size (used only when *bins* is ``None``).
        genomes_table: Optional path to a genomes metadata TSV.  One column
            holds the genome names (see *genomes_table_format*); every other
            column is stored in the ``/genomes`` table.
        genomes_table_format: Which column of *genomes_table* holds the genome
            names: a 1-based position or a header name, plus ``,noheader`` for
            a table without a header line.  Defaults to ``genome_name``.
        pairs_kwargs: Extra kwargs passed to the pairs reader (column
            indices, etc.), the same for every pairs file.
        metadata: Additional metadata dict stored in file attrs.
        chunksize: Number of *pairs* to process per chunk.  This is chunk
            granularity, not read memory: pyarrow's read block is capped
            separately at ``_PAIRS_READ_BLOCK_MAX_BYTES``, and the binned
            batches are coalesced back up to *chunksize* afterwards.  Under
            ``sort_in_memory=False`` every chunk costs one ``cooler.create``
            call, whose runtime is dominated by a fixed overhead that scales
            with contig count rather than pixels (seconds per call on a
            metagenome), so small chunks are disproportionately expensive.
            The cost of a large value is the pixels held while coalescing one
            chunk — roughly ``chunksize`` rows at 24 bytes in the worst case.
        sort_in_memory: If ``True``, collect all pixels in RAM and sort before
            writing (fast for small files).  If ``False``, pass the unsorted
            stream to cooler's external sort (scalable for large files).  If
            ``None`` (default), buffer the binned pixels and use ``True`` unless
            they exceed 100 M rows, in which case fall back to ``False`` with a
            logged warning.
        qc: Tally contact-quality statistics while the pairs are being binned
            and store them in ``metadata["qc"]`` and the ``qc_*`` columns of
            ``/genomes`` and ``/contigs`` (see :mod:`metacooler.qc`).  Free
            apart from a few vectorised operations per chunk, since the pass
            over the pairs is happening anyway — and only possible *here*,
            because binning discards the sub-bin distances the short/long
            split needs.  The per-contig short/long counts are what lets a
            later :func:`metacooler.io.reassign.reassign` keep that split.
        qc_distance_threshold: Intra-contig short/long split for *qc*, in bp.
        ps: Accumulate a per-genome P(s) curve from the same pass and write it
            beside the output as ``<name>.ps.tsv`` (see :mod:`metacooler.ps`).
            Measured on read positions, so it resolves distances the binned map
            cannot — which is the whole reason to do it here rather than
            afterwards.  Costs one ``searchsorted`` and one ``bincount`` per
            chunk over the intra-contig rows.
        ps_bins_per_decade: Log-grid resolution of the P(s) curve.
        force: Overwrite an existing ``<name>.ps.tsv``.  The ``.metacool``
            itself is guarded by the caller (the CLI checks it before any work
            starts).
        nproc: Threads binning pairs batches while the file is being read and
            decompressed (which stays on one thread, being one gzip stream).
            Results are folded in file order, so the output and every tally
            are those of ``nproc=1``; memory grows by one batch per thread.
        keep_partial: On a failure while the output is being written, leave
            ``<uri>.partial`` behind for inspection, as the default does
            (logged at ERROR).  ``False`` removes it instead, whatever stopped
            the run (``KeyboardInterrupt`` included), and logs that at INFO.
            The output path itself is never written by a failed run, nor is
            the ``.ps.tsv`` table, staged as ``<name>.ps.tsv.partial`` and
            moved into place just before the map: the previous table is set
            aside (as ``<name>.ps.tsv.previous``) until the map is in place,
            and put back if the map cannot be.  Once the map is in place the
            set-aside table is removed; if that fails it is left, with a
            WARNING, and not put back: the new map and table match, and the
            run succeeds unless it was interrupted.  Only a process killed
            during those moves can leave them half done: the previous table
            still set aside, or the new table beside the previous map.

    Raises:
        ValueError: If *nproc* is below 1, no pairs file is given, the pairs
            files are not compatible (see *pairs*), or *contig_genome_map*
            names no contig of the assembly (checked before any pair is read).
        FileNotFoundError: If a pairs file does not exist -- checked before
            any is read, so a typo in the last of several libraries does not
            cost a binning pass over the others.
        FileExistsError: If ``<name>.ps.tsv.previous`` exists, with or
            without *ps*.  A run leaves it when it was killed while it moved
            its map and table into place, or could not remove that table
            afterwards, and which table matches the map is for the user to
            check; a file of the user's own of that name is refused alike.
            Checked before anything is read.
    """
    if nproc < 1:
        raise ValueError(f"nproc must be at least 1, got {nproc}")
    uri = Path(uri)
    if pairs_kwargs is None:
        pairs_kwargs = {}
    pairs_paths = _pairs_paths(pairs)
    missing = [str(p) for p in pairs_paths if not p.exists()]
    if missing:
        raise FileNotFoundError(f"Pairs file(s) not found: {', '.join(missing)}")
    if len(pairs_paths) > 1:
        logger.info(
            "Reading %d pairs files in one pass: %s",
            len(pairs_paths),
            ", ".join(map(str, pairs_paths)),
        )
    if contig_genome_map is not None and not isinstance(
        contig_genome_map, dict | str | Path
    ):
        raise TypeError("contig_genome_map must be a dict, a path, or None")
    if bins is not None and binsize is not None:
        # Passing both silently binned every contact arithmetically against a bin
        # table that was never tiled at *binsize*, scattering contacts across
        # unrelated fragments while still passing validate().
        raise ValueError(
            "binsize and bins are mutually exclusive: pass binsize to tile "
            "fixed-size bins, or bins to use a pre-defined bin table."
        )
    _check_no_binning(
        contig_genome_map,
        contig_table_format,
        genomes_table=genomes_table,
        drop_unassigned=drop_unassigned,
    )
    # A P(s) table still set aside: an earlier run was killed while it moved its
    # map and table into place, and which table matches the map is unknown.
    _refuse_a_set_aside_file(uri, (ps_table_path(uri),))
    # Guard the P(s) sidecar before the streaming pass, not after it: finding
    # out that the table cannot be written is worth seconds, not hours.
    ps_path = ps_table_path(uri) if ps else None
    if ps_path is not None:
        check_output_path(ps_path, force, defer_removal=True)
    inputs = [
        *pairs_paths,
        bins if isinstance(bins, str | Path) else None,
        fasta,
        contig_genome_map if isinstance(contig_genome_map, str | Path) else None,
        genomes_table,
    ]
    companions = () if ps_path is None else (ps_path,)
    _refuse_staging_an_input(uri, inputs, companions)
    timer = time.perf_counter()

    # --- Parse contig map and optional genomes metadata table ---
    cg_map, contig_df = _parse_contig_map(contig_genome_map, contig_table_format)
    extra_genomes_df = _load_extra_genomes_table(genomes_table, genomes_table_format)

    # --- Resolve contig lengths and initial bins DataFrame ---
    if isinstance(bins, str | Path):
        bins_df = _read_bins_bed(bins)
        contig_lengths = _read_contig_lengths(bins=bins_df)
        assembly_source = f"the bins file {bins}"
    elif isinstance(bins, pd.DataFrame):
        bins_df = bins.copy()
        contig_lengths = _read_contig_lengths(bins=bins_df)
        assembly_source = "the bins table"
    elif bins is None:
        # Integer binsize mode: need contig lengths from fasta or pairs header
        contig_lengths = _read_contig_lengths(fasta=fasta, pairs=pairs_paths)
        assembly_source = (
            f"the FASTA {fasta}" if fasta is not None else "the pairs header(s)"
        )
        bins_df = None
    else:
        raise TypeError("bins must be a path, DataFrame, or None")

    # --- Check the contig table against the assembly, before a pair is read ---
    # Against every contig the assembly names: non-positive lengths,
    # --drop-unassigned and --filter only narrow the set afterwards.  No table
    # at all is no binning yet, and has nothing to check.
    if contig_genome_map is not None:
        _check_contig_table_matches(
            cg_map,
            contig_lengths,
            source=contig_genome_map,
            contig_table_format=contig_table_format,
            assembly_source=assembly_source,
        )

    # --- Resolve the reference ("fasta") contig order, if one is requested ---
    # When lengths come from a bins table their dict order is whatever
    # groupby("chrom") produced (alphabetical), not the FASTA order.
    if contig_ordering == "fasta" and bins_df is not None:
        contig_lengths = _apply_reference_order(
            contig_lengths, fasta=fasta, pairs=pairs_paths
        )

    contig_lengths, bins_df = _drop_nonpositive_contigs(contig_lengths, bins_df)
    logger.info(f"Determined lengths for {len(contig_lengths)} contigs")

    # --- Filter unassigned contigs ---
    if drop_unassigned:
        n_before = len(contig_lengths)
        contig_lengths = {
            ct: ls
            for ct, ls in contig_lengths.items()
            if ct in cg_map and cg_map[ct] != "__unbinned__"
        }
        if bins_df is not None:
            bins_df = bins_df[bins_df["chrom"].isin(contig_lengths)].copy()
        logger.info(
            f"Dropped {n_before - len(contig_lengths)} unassigned contigs; "
            f"{len(contig_lengths)} remain"
        )
        _require_assigned_contigs(contig_lengths, n_before)

    # --- Apply --filter predicates before anything is tiled or binned ---
    contig_lengths, bins_df = _apply_create_filters(
        contig_lengths,
        cg_map,
        contig_df,
        extra_genomes_df,
        filters,
        bins_df,
    )

    # --- Build canonical contig order and genome / contig tables ---
    contig_order = _build_contig_order(contig_lengths, cg_map, ordering=contig_ordering)
    logger.debug(f"Contig order: {contig_order[:10]} ... {contig_order[-10:]}")

    genomes_df = _build_genomes_table(
        contig_order,
        contig_lengths,
        cg_map,
        extra_genomes_df,
    )
    contig_name_to_id = {name: i for i, name in enumerate(contig_order)}
    contigs_df = _build_contigs_table(
        contig_order, contig_lengths, cg_map, genomes_df, contig_df
    )
    # Before the pairs are streamed: a name cooler cannot read costs a second
    # here and an entire binning pass if it is only caught at write time.
    check_ascii_names(genomes_df, contigs_df)
    logger.info(
        f"Built final genomes and contigs tables with {len(genomes_df)} genomes "
        f"and {len(contigs_df)} contigs"
    )

    # --- Build final bins table ---
    if bins_df is None:
        if binsize is None:
            raise ValueError("binsize required when bins is None")
        bins_df = _tile_bins(contig_order, contig_lengths, contig_name_to_id, binsize)
    else:
        bins_df = _build_bins(bins_df, contig_name_to_id)
    bins_df = bins_df.sort_values(["chrom", "start"]).reset_index(drop=True)
    logger.info(
        f"Built final bins table with {len(bins_df)} bins tiling "
        f"{len(contig_lengths)} contigs"
    )

    # --- Compute chrom / genome offsets and insert n_bins into genomes_df ---
    chrom_offset, genome_offset = _compute_all_offsets(bins_df, contigs_df, genomes_df)

    # Everything the tables were built from is dead now, and the binning pass
    # below is the memory-hungry one: `_buffer_pixels` accumulates up to
    # _SORT_IN_MEMORY_PIXEL_THRESHOLD rows and folds them, which transiently
    # doubles. Holding these on for its whole duration costs gigabytes on a
    # metagenome -- `contig_df` alone is the user's text table with one Python
    # str per cell (~1.5 GiB at 1.9 M contigs x 10 columns, measured).
    del contig_df, cg_map, extra_genomes_df, contig_lengths

    # --- Stream pairs, bin, sort, and write the cooler file ---
    logger.info(f"Starting to create cooler file at {uri}...")
    bin_keys: np.ndarray | None = (
        None
        if binsize is not None
        else _folded_bin_keys(chrom_offset, bins_df["start"].to_numpy())
    )
    drop_stats = _PairsDropStats()
    qc_tally = (
        ContactQC(
            len(genomes_df),
            qc_distance_threshold,
            source="pairs",
            unbinned_genome=unbinned_genome_id(genomes_df["name"]),
            n_contigs=len(contigs_df),
        )
        if qc
        else None
    )
    contig_genome_id = contigs_df["genome_id"].to_numpy(dtype=np.int64)
    contig_length = contigs_df["length"].to_numpy(dtype=np.int64)
    ps_edges = pairs_distance_grid(contig_length, ps_bins_per_decade) if ps else None
    ps_tally = PSCounts(len(genomes_df), ps_edges) if ps_edges is not None else None
    raw_pixels: Iterable[pd.DataFrame] = _stream_binned_pairs(
        pairs_paths,
        contig_name_to_id,
        chrom_offset,
        binsize,
        bin_keys,
        chunksize,
        pairs_kwargs,
        drop_stats,
        qc_tally,
        ps_tally,
        contig_genome_id if (qc or ps) else None,
        contig_length,
        nproc=nproc,
    )
    pixel_source: Iterable[pd.DataFrame]
    if sort_in_memory is None:
        # Decide from the pixel volume actually produced, not from a proxy such
        # as the pairs file size: binning aggregates duplicates away, so even a
        # multi-GiB pairs file usually yields few enough pixels to sort in RAM —
        # which writes the cooler in a single pass instead of one temporary
        # cooler per chunk.
        frames, rest = _buffer_pixels(raw_pixels, _SORT_IN_MEMORY_PIXEL_THRESHOLD)
        if rest is None:
            pixel_source = _sort_and_dedup_pixels(pixel_iter=frames)
            ordered = True
        else:
            logger.warning(
                "Deduplicated pixels exceeded the %s-row in-memory budget "
                "(%s distinct pixels buffered so far) — switching to "
                "sort_in_memory=False (cooler external sort). Pass "
                "sort_in_memory=True to override.",
                f"{_SORT_IN_MEMORY_PIXEL_THRESHOLD:,}",
                f"{len(frames[0]):,}" if frames else "0",
            )
            pixel_source = _chain_pixel_frames(frames, rest)
            ordered = False
    elif sort_in_memory:
        pixel_source = _sort_and_dedup_pixels(pixel_iter=raw_pixels)
        ordered = True
    else:
        pixel_source = raw_pixels
        ordered = False
    pixel_source, was_empty = _ensure_nonempty_pixel_stream(pixel_source)
    ordered = ordered or was_empty
    # The map and its P(s) sidecar are staged together, and both moved into
    # place only once both are written and the map has validated.
    with _staged_output(
        uri, keep_partial=keep_partial, inputs=inputs, companions=companions
    ) as staged:
        write_cooler(
            staged,
            bins_df,
            pixel_source,
            binsize=binsize,
            ordered=ordered,
            symmetric_upper=True,
            assembly=assembly_name,
            metadata=metadata or {},
            dupcheck=False,
        )

        # --- Post-process: add metacooler-specific groups and attrs ---
        logger.info("Post-processing cooler file to add metacool structure")
        _finalize_metacool_file(
            staged,
            genomes_df,
            contigs_df,
            chrom_offset,
            genome_offset,
            contig_ordering=contig_ordering,
            assembly_name=assembly_name,
            binsize=binsize,
        )

        # --- Contact-quality statistics ---
        # Written here rather than through write_cooler's `metadata`: the tally
        # is only complete once the pixel stream above has been consumed, which
        # happens inside that very call.
        if qc_tally is not None:
            store_qc(
                staged,
                qc_tally.summary(
                    binsize=binsize,
                    n_pairs_read=drop_stats.total,
                    n_pairs_discarded=drop_stats.dropped,
                ),
                qc_tally.genome_table(genomes_df["name"].to_numpy()),
                qc_tally.contig_table(contigs_df["name"].to_numpy()),
            )

        # --- Validate ---
        logger.info("Validating created metacool file")
        validate(staged)

        # --- P(s) sidecar: a companion file, not part of the .metacool ---
        if ps_tally is not None and ps_edges is not None and ps_path is not None:
            write_ps_table(
                _partial_path(ps_path),
                assemble(
                    genomes_df["name"].to_numpy(),
                    ps_edges,
                    ps_tally.matrix,
                    position_pair_denominators(
                        contig_length, contig_genome_id, len(genomes_df), ps_edges
                    ),
                ),
                force=True,
            )

    with h5py.File(uri, "r") as f:
        nnz = int(f.attrs.get("nnz"))
        total = int(f.attrs.get("sum", 0))
    n_bins = len(bins_df)
    n_contigs = len(contigs_df)
    n_genomes = len(genomes_df)
    logger.info(
        f"Created {n_bins} bins from {n_contigs} contigs in {n_genomes} genomes. "
        f"NNZ={nnz}, sum={total}.",
    )
    logger.info(f"Created metacool file: {uri} ({_total_time_str(timer)} elapsed)")

from_cool

from_cool(
    cool_uri: str | Path,
    output: str | Path,
    contig_genome_map: str | Path | None = None,
    contig_table_format: str = CONTIGS_FORMAT_DEFAULT,
    genomes_table: str | Path | None = None,
    contig_ordering: Literal[
        "genome-then-length", "genome-then-alpha", "cool", "alpha"
    ] = "genome-then-length",
    assembly_name: str | None = None,
    drop_unassigned: bool = False,
    filters: Sequence[str] | None = None,
    metadata: dict | None = None,
    nproc: int = 1,
    sort_in_memory: bool | None = None,
    qc: bool = True,
    qc_distance_threshold: int = DEFAULT_DISTANCE_THRESHOLD,
    genomes_table_format: str = GENOMES_FORMAT_DEFAULT,
    keep_partial: bool = True,
) -> None

Create a metacool file from an existing cool file.

Parameters:

Name Type Description Default
cool_uri str | Path

Path to the source cool file.

required
output str | Path

Destination path for the output metacool file.

required
contig_genome_map str | Path | None

Path to contig-to-genome assignment table (str or Path only; dicts are not accepted here). Contigs it does not list are collected under __unbinned__, and names that are not contigs of the cool file are ignored with a warning, but it must name at least one contig of the cool file. None (the default) builds a map with no binning yet: the cool file plus the genome layer, every contig in __unbinned__ (bin them later with reassign; contig_ordering="cool" keeps the cool file's own contig order, and makes that reassignment cheap).

None
contig_table_format str

Which columns of the table hold the contig and the genome names: CONTIG,GENOME[,noheader], each a 1-based position or a header name, or an alias (see :func:metacooler.io.contigs.parse_contigs_table).

CONTIGS_FORMAT_DEFAULT
genomes_table str | Path | None

Optional path to genomes metadata table.

None
contig_ordering Literal['genome-then-length', 'genome-then-alpha', 'cool', 'alpha']

Strategy for ordering contigs within the metacool file. 'cool' preserves the original contig order from the input cool file. Other choices: 'genome-then-length', 'genome-then-alpha', 'alpha'.

'genome-then-length'
assembly_name str | None

Name for this assembly / metagenome, stored in file attrs.

None
drop_unassigned bool

If True, drop contigs absent from the contig table instead of collecting them under __unbinned__.

False
filters Sequence[str] | None

--filter expressions (see :mod:metacooler.filters) applied to the derived /genomes and /contigs tables before the bins are remapped. A dropped contig keeps none of its bins, and every pixel touching one is discarded. genomes.n_bins is not available here.

None
metadata dict | None

Optional metadata dictionary to store in the file attrs.

None
nproc int

Accepted for backwards compatibility, but currently has no effect: HDF5 reads must be serialised, so pixel remapping runs in a single thread. Values greater than 1 log a warning.

1
sort_in_memory bool | None

If True, collect all pixels in RAM and sort before writing (fast for small files). If False, pass the unsorted stream to cooler's external sort (scalable for large files). If None (default), auto-detect based on pixel count: files with more than 100 M pixels use False with a logged warning.

None
qc bool

Tally contact-quality statistics off the pixel stream that is being copied anyway, and store them in metadata["qc"] and the qc_* columns of /genomes and /contigs (see :mod:metacooler.qc). A cool file has no read positions left, so intra-contig distances are measured between bin midpoints and the block records source="pixels".

True
qc_distance_threshold int

Intra-contig short/long split for qc, in bp.

DEFAULT_DISTANCE_THRESHOLD
genomes_table_format str

Which column of genomes_table holds the genome names: a 1-based position or a header name, plus ,noheader for a table without a header line. Defaults to genome_name.

GENOMES_FORMAT_DEFAULT
keep_partial bool

On a failure while the output is being written, leave <output>.partial behind for inspection, as the default does (logged at ERROR). False removes it instead, whatever stopped the run (KeyboardInterrupt included), and logs that at INFO. The output path itself is never written by a failed run.

True

Raises:

Type Description
ValueError

If cool_uri is not a valid cooler file, or contig_genome_map names no contig of it (checked before any pixel is read).

Source code in src/metacooler/io/create.py
3236
3237
3238
3239
3240
3241
3242
3243
3244
3245
3246
3247
3248
3249
3250
3251
3252
3253
3254
3255
3256
3257
3258
3259
3260
3261
3262
3263
3264
3265
3266
3267
3268
3269
3270
3271
3272
3273
3274
3275
3276
3277
3278
3279
3280
3281
3282
3283
3284
3285
3286
3287
3288
3289
3290
3291
3292
3293
3294
3295
3296
3297
3298
3299
3300
3301
3302
3303
3304
3305
3306
3307
3308
3309
3310
3311
3312
3313
3314
3315
3316
3317
3318
3319
3320
3321
3322
3323
3324
3325
3326
3327
3328
3329
3330
3331
3332
3333
3334
3335
3336
3337
3338
3339
3340
3341
3342
3343
3344
3345
3346
3347
3348
3349
3350
3351
3352
3353
3354
3355
3356
3357
3358
3359
3360
3361
3362
3363
3364
3365
3366
3367
3368
3369
3370
3371
3372
3373
3374
3375
3376
3377
3378
3379
3380
3381
3382
3383
3384
3385
3386
3387
3388
3389
3390
3391
3392
3393
3394
3395
3396
3397
3398
3399
3400
3401
3402
3403
3404
3405
3406
3407
3408
3409
3410
3411
3412
3413
3414
3415
3416
3417
3418
3419
3420
3421
3422
3423
3424
3425
3426
3427
3428
3429
3430
3431
3432
3433
3434
3435
3436
3437
3438
3439
3440
3441
3442
3443
3444
3445
3446
3447
3448
3449
3450
3451
3452
3453
3454
3455
3456
3457
3458
3459
3460
3461
3462
3463
3464
3465
3466
3467
3468
3469
3470
3471
3472
3473
3474
3475
3476
3477
3478
3479
3480
3481
3482
3483
3484
3485
3486
3487
3488
3489
3490
3491
3492
3493
3494
3495
3496
3497
3498
3499
3500
3501
3502
3503
3504
3505
3506
3507
3508
3509
3510
3511
3512
3513
3514
3515
3516
3517
3518
3519
3520
3521
3522
3523
3524
3525
3526
3527
3528
3529
3530
3531
3532
3533
3534
3535
3536
3537
3538
3539
3540
3541
3542
3543
3544
3545
3546
def from_cool(
    cool_uri: str | Path,
    output: str | Path,
    contig_genome_map: str | Path | None = None,
    contig_table_format: str = CONTIGS_FORMAT_DEFAULT,
    genomes_table: str | Path | None = None,
    contig_ordering: Literal[
        "genome-then-length", "genome-then-alpha", "cool", "alpha"
    ] = "genome-then-length",
    assembly_name: str | None = None,
    drop_unassigned: bool = False,
    filters: Sequence[str] | None = None,
    metadata: dict | None = None,
    nproc: int = 1,
    sort_in_memory: bool | None = None,
    qc: bool = True,
    qc_distance_threshold: int = DEFAULT_DISTANCE_THRESHOLD,
    genomes_table_format: str = GENOMES_FORMAT_DEFAULT,
    keep_partial: bool = True,
) -> None:
    """Create a metacool file from an existing cool file.

    Args:
        cool_uri: Path to the source cool file.
        output: Destination path for the output metacool file.
        contig_genome_map: Path to contig-to-genome assignment table
            (``str`` or ``Path`` only; dicts are not accepted here).  Contigs
            it does not list are collected under ``__unbinned__``, and names
            that are not contigs of the cool file are ignored with a warning,
            but it must name at least one contig of the cool file.  ``None``
            (the default) builds a map with no binning yet: the cool file plus
            the genome layer, every contig in ``__unbinned__`` (bin them later
            with ``reassign``; ``contig_ordering="cool"`` keeps the cool
            file's own contig order, and makes that reassignment cheap).
        contig_table_format: Which columns of the table hold the contig and
            the genome names: ``CONTIG,GENOME[,noheader]``, each a 1-based
            position or a header name, or an alias (see
            :func:`metacooler.io.contigs.parse_contigs_table`).
        genomes_table: Optional path to genomes metadata table.
        contig_ordering: Strategy for ordering contigs within the metacool file.
            ``'cool'`` preserves the original contig order from the input cool file.
            Other choices: ``'genome-then-length'``, ``'genome-then-alpha'``,
            ``'alpha'``.
        assembly_name: Name for this assembly / metagenome, stored in file attrs.
        drop_unassigned: If ``True``, drop contigs absent from the contig table
            instead of collecting them under ``__unbinned__``.
        filters: ``--filter`` expressions (see :mod:`metacooler.filters`)
            applied to the derived ``/genomes`` and ``/contigs`` tables before
            the bins are remapped.  A dropped contig keeps none of its bins,
            and every pixel touching one is discarded.  ``genomes.n_bins`` is
            not available here.
        metadata: Optional metadata dictionary to store in the file attrs.
        nproc: Accepted for backwards compatibility, but currently has **no
            effect**: HDF5 reads must be serialised, so pixel remapping runs in
            a single thread.  Values greater than 1 log a warning.
        sort_in_memory: If ``True``, collect all pixels in RAM and sort before
            writing (fast for small files).  If ``False``, pass the unsorted
            stream to cooler's external sort (scalable for large files).  If
            ``None`` (default), auto-detect based on pixel count: files with
            more than 100 M pixels use ``False`` with a logged warning.
        qc: Tally contact-quality statistics off the pixel stream that is
            being copied anyway, and store them in ``metadata["qc"]`` and the
            ``qc_*`` columns of ``/genomes`` and ``/contigs`` (see
            :mod:`metacooler.qc`).  A cool file has no read positions left, so
            intra-contig distances are measured between bin midpoints and the
            block records ``source="pixels"``.
        qc_distance_threshold: Intra-contig short/long split for *qc*, in bp.
        genomes_table_format: Which column of *genomes_table* holds the genome
            names: a 1-based position or a header name, plus ``,noheader`` for
            a table without a header line.  Defaults to ``genome_name``.
        keep_partial: On a failure while the output is being written, leave
            ``<output>.partial`` behind for inspection, as the default does
            (logged at ERROR).  ``False`` removes it instead, whatever stopped
            the run (``KeyboardInterrupt`` included), and logs that at INFO.
            The output path itself is never written by a failed run.

    Raises:
        ValueError: If ``cool_uri`` is not a valid cooler file, or
            *contig_genome_map* names no contig of it (checked before any
            pixel is read).
    """
    cool_path = Path(cool_uri)
    output_path = Path(output)
    inputs = [cool_uri, contig_genome_map, genomes_table]
    _refuse_staging_an_input(output_path, inputs)
    timer = time.perf_counter()
    _check_no_binning(
        contig_genome_map,
        contig_table_format,
        genomes_table=genomes_table,
        drop_unassigned=drop_unassigned,
    )

    # --- Validate and load cool file ---
    if not cooler.fileops.is_cooler(str(cool_path)):
        raise ValueError(f"Not a valid cooler file: {cool_path}")
    clr = cooler.Cooler(str(cool_path))
    # A metacool is symmetric-upper, and the remap below folds every pixel into
    # the upper triangle. On a "square" cooler -- which stores both triangles --
    # that sums each off-diagonal contact with its mirror, doubling it, and the
    # result still validates. Folding is only correct for a symmetric matrix,
    # and square storage is chosen precisely when one is not guaranteed.
    storage_mode = clr.info.get("storage-mode", "symmetric-upper")
    if storage_mode != "symmetric-upper":
        raise ValueError(
            f"{cool_path} uses storage-mode {storage_mode!r}; metacooler only "
            "imports symmetric-upper coolers (the cooler default). Re-create "
            "it with symmetric_upper=True, keeping the upper triangle."
        )
    bins = pd.DataFrame(clr.bins()[:])
    n_pixels = clr.info["nnz"]
    logger.info(f"Loaded cool file: {len(bins)} bins, {n_pixels} pixels")

    # --- Auto-detect sort_in_memory based on pixel count ---
    if sort_in_memory is None:
        if n_pixels > _SORT_IN_MEMORY_PIXEL_THRESHOLD:
            logger.warning(
                "Input has %d pixels — switching to sort_in_memory=False "
                "(cooler external sort). Pass sort_in_memory=True to override.",
                n_pixels,
            )
            sort_in_memory = False
        else:
            sort_in_memory = True

    # Build contig lengths from bins (use actual contigs in bins, not chroms table).
    # One vectorised max per contig, not one materialised sub-frame per contig:
    # ``clr.bins()[:]`` hands back ``chrom`` as a Categorical, so iterating the
    # groupby builds a DataFrame per contig (~14 us each, measured) and costs
    # half a minute on a metagenome before a single pixel has been read.
    # ``observed=True`` also drops the empty groups a zero-length chrom would
    # otherwise contribute as a NaN max, which ``int()`` cannot convert.
    chroms = clr.chroms()[:]
    # Every contig of the cool file, keyed by name in the file's order: the
    # "cool" ordering preserves it, and the contig table is checked against it.
    cool_contigs = dict.fromkeys(chroms["name"])
    _max_end = bins.groupby("chrom", observed=True)["end"].max()
    contig_lengths: dict[str, int] = {
        str(name): int(end)
        for name, end in zip(_max_end.index, _max_end.to_numpy(), strict=True)
    }
    del _max_end
    contig_lengths, _ = _drop_nonpositive_contigs(contig_lengths)

    # --- Parse contig assignments ---
    contig_map, contig_df = _parse_contig_map(contig_genome_map, contig_table_format)
    # Against every contig of the cool file, before --drop-unassigned and
    # --filter narrow them, and before a single pixel is read.  No table at all
    # is no binning yet, and has nothing to check.
    if contig_genome_map is not None:
        _check_contig_table_matches(
            contig_map,
            cool_contigs,
            source=contig_genome_map,
            contig_table_format=contig_table_format,
            assembly_source=f"the cool file {cool_path}",
        )
    extra_genomes_df = _load_extra_genomes_table(genomes_table, genomes_table_format)

    # --- Filter unassigned contigs ---
    if drop_unassigned:
        n_before = len(contig_lengths)
        contig_lengths = {
            ct: ln
            for ct, ln in contig_lengths.items()
            if ct in contig_map and contig_map[ct] != "__unbinned__"
        }
        _require_assigned_contigs(contig_lengths, n_before)

    # --- Apply --filter predicates before the bins are remapped ---
    contig_lengths, _ = _apply_create_filters(
        contig_lengths, contig_map, contig_df, extra_genomes_df, filters
    )

    # --- Build contig order ---
    if contig_ordering == "cool":
        # Preserve the original contig order from the cool file's chroms table,
        # filtered to only contigs present in contig_lengths (after drop_unassigned).
        contig_order = [name for name in cool_contigs if name in contig_lengths]
    else:
        contig_order = _build_contig_order(
            contig_lengths=contig_lengths,
            contig_genome_map=contig_map,
            ordering=contig_ordering,
        )

    # --- Build genome / contig tables ---
    genomes = _build_genomes_table(
        contig_order=contig_order,
        contig_lengths=contig_lengths,
        contig_genome_map=contig_map,
        extra_genomes_df=extra_genomes_df,
    )
    contigs = _build_contigs_table(
        contig_order=contig_order,
        contig_lengths=contig_lengths,
        contig_genome_map=contig_map,
        genomes=genomes,
        contig_df=contig_df,
    )
    # Fail before the pixel stream, not after it (see from_pairs).
    check_ascii_names(genomes, contigs)

    # --- Remap bins to new contig IDs; compute offsets ---
    bins_remapped, old_to_new_bin = _remap_bins(bins, contig_order)
    chrom_offset, genome_offset = _compute_all_offsets(bins_remapped, contigs, genomes)

    # Free the sources the tables were derived from before the pixel stream, for
    # the reason given in from_pairs: the streaming pass is where the memory is
    # wanted, and none of these is read again.
    del bins, chroms, cool_contigs, contig_df, contig_map, extra_genomes_df
    del contig_lengths, contig_order

    # --- Stream, remap, sort, and write ---
    logger.info(
        f"Building metacool file: {len(genomes)} genomes, {len(contigs)} contigs"
    )
    # Preserve the source count width: hard-casting to int32 wraps the int64
    # counts ops/binnify.py deliberately writes, and floors float counts to 0.
    count_dtype = resolve_count_dtype(
        clr.pixels()[0:1]["count"].dtype, context="from_cool"
    )
    raw_pixels: Iterable[pd.DataFrame] = _stream_remapped_pixels(
        str(cool_path), clr.info["nnz"], old_to_new_bin, nproc, count_dtype
    )
    qc_tally = (
        ContactQC(
            len(genomes),
            qc_distance_threshold,
            source="pixels",
            unbinned_genome=unbinned_genome_id(genomes["name"]),
            n_contigs=len(contigs),
        )
        if qc
        else None
    )
    if qc_tally is not None:
        raw_pixels = _tally_pixel_qc(
            raw_pixels, qc_tally, bin_maps(bins_remapped, contigs)
        )
    if sort_in_memory:
        # Contigs are only reordered -- bins map one-to-one -- so the remapped
        # pixels stay unique and this is one argsort on the pixel key: the
        # reduction after it finds no repeated key and returns the counts as
        # they are.  It is kept because a cooler written without cooler's
        # duplicate check can repeat a pixel, which is then summed.
        pixel_source: Iterable[pd.DataFrame] = _sort_and_dedup_pixels(
            pixel_iter=raw_pixels
        )
        ordered = True
    else:
        pixel_source: Iterable[pd.DataFrame] = raw_pixels
        ordered = False
    pixel_source, was_empty = _ensure_nonempty_pixel_stream(pixel_source)
    ordered = ordered or was_empty
    with _staged_output(
        output_path,
        keep_partial=keep_partial,
        inputs=inputs,
    ) as staged:
        write_cooler(
            staged,
            bins_remapped,
            pixel_source,
            ordered=ordered,
            symmetric_upper=True,
            # Keep the source's assembly name unless the caller overrides it.
            assembly=assembly_name or _source_assembly(clr) or "imported",
            metadata=metadata or {},
            dtypes={"count": count_dtype},
        )

        # --- Add metacooler-specific groups and attributes ---
        logger.info("Post-processing cooler file to add metacool structure")
        _finalize_metacool_file(
            staged,
            genomes,
            contigs,
            chrom_offset,
            genome_offset,
            contig_ordering=contig_ordering,
            assembly_name=assembly_name,
        )

        # --- Contact-quality statistics ---
        # Complete only now: the tally rides on the pixel stream that
        # write_cooler above consumed.
        if qc_tally is not None:
            store_qc(
                staged,
                qc_tally.summary(binsize=infer_binsize(bins_remapped)),
                qc_tally.genome_table(genomes["name"].to_numpy()),
                qc_tally.contig_table(contigs["name"].to_numpy()),
            )

        # --- Validate ---
        logger.info("Validating created metacool file")
        validate(staged)
    with h5py.File(output_path, "r") as f:
        nnz = int(f.attrs.get("nnz"))
        total = int(f.attrs.get("sum"))
        n_bins = int(f.attrs.get("nbins"))
        n_contigs = int(f.attrs.get("nchroms"))
        n_genomes = int(f.attrs.get("ngenomes"))
    logger.info(
        f"Created {n_bins} bins from {n_contigs} contigs in {n_genomes} genomes. "
        f"NNZ={nnz}, sum={total}.",
    )
    logger.info(
        f"Created metacool file: {output_path} ({_total_time_str(timer)} elapsed)"
    )

Collecting single-genome cool files

Collect single-genome cool files into one metacool file.

Each input — a .cool file, or one resolution of an .mcool addressed as file.mcool::resolutions/<binsize> — becomes one genome of the output, named after the input's file stem unless names says otherwise. Every contig is renamed <genome>__<contig>, so the chromosome or contig_1 that so many assemblies share cannot collide.

The layout is what makes this cheap. Contigs are always grouped by genome, in genome_id order, so every input occupies one contiguous block of output bins. An input's pixels only ever address its own block — there is nothing between genomes to address — so the output pixel table is the concatenation of the inputs' tables, each shifted to its block. Under the default contig_ordering="cool", which keeps every input's own contig order, each shifted table is still sorted, and the whole collection is written in one streaming pass with no sort at all. Only an ordering that reorders contigs within a genome (genome-then-length/genome-then-alpha) forces a re-sort, and then only of the inputs it actually reordered, one at a time.

The corollary is worth saying out loud: the inter-genome blocks of a collected map are empty by construction. The inputs were binned separately, usually from separate libraries, so no contact between two of them was ever observed.

CONTIG_NAME_SEPARATOR module-attribute

CONTIG_NAME_SEPARATOR = '__'

COLLECT_ORDERINGS module-attribute

COLLECT_ORDERINGS = ('cool', 'genome-then-length', 'genome-then-alpha')

collect

collect(
    inputs: Sequence[str | Path],
    output: str | Path,
    names: Sequence[str] | None = None,
    genomes_table: str | Path | None = None,
    contigs_table: str | Path | None = None,
    contig_ordering: CollectOrdering = "cool",
    assembly_name: str | None = None,
    filters: Sequence[str] | None = None,
    metadata: dict | None = None,
    sort_in_memory: bool | None = None,
    chunksize: int = DEFAULT_CHUNKSIZE,
    qc: bool = True,
    qc_distance_threshold: int = DEFAULT_DISTANCE_THRESHOLD,
    genomes_table_format: str = GENOMES_FORMAT_DEFAULT,
    contigs_table_format: str = CONTIG_METADATA_FORMAT_DEFAULT,
    keep_partial: bool = True,
) -> None

Collect single-genome cool files into one metacool file, one genome each.

Every input becomes one genome of the output, named after its file stem (E_coli.cool → E_coli) or by names. Every contig is renamed <genome>__<contig> (see :func:collected_contig_name), so contig names shared between assemblies cannot collide. Bins and pixels are carried over unchanged; contacts between two inputs were never observed, so the output's inter-genome blocks are empty.

Parameters:

Name Type Description Default
inputs Sequence[str | Path]

Cool files, or cooler URIs such as file.mcool::resolutions/1000. All must share one bin size, or all have variable (e.g. restriction-fragment) bins. Metacool files are refused: each input must be a single genome.

required
output str | Path

Destination .metacool path. Written to a .partial sibling first and moved into place once it validates.

required
names Sequence[str] | None

Genome name for each input, in order. Defaults to the file stems, which must then be unique.

None
genomes_table str | Path | None

Optional genomes metadata TSV. Its genome-name column (see genomes_table_format) is matched against the genome names; every other column is stored in /genomes. A mismatch in either direction is reported as a warning.

None
contigs_table str | Path | None

Optional per-contig metadata TSV. Its contig-name column (see contigs_table_format) names a contig as in its input cool or as collected; every other column is stored in /contigs. An optional genome_name column says which genome each row describes — it disambiguates names several inputs share, and is checked, never used to move a contig to another genome.

None
contig_ordering CollectOrdering

"cool" (default) keeps every input's own contig order; "genome-then-length" puts each genome's longest contig first; "genome-then-alpha" sorts each genome's contigs by name. Genomes are always laid out in genome_id (alphabetical) order. "cool" never needs a pixel sort.

'cool'
assembly_name str | None

Stored in the genome-assembly attribute ("unknown" when not given).

None
filters Sequence[str] | None

--filter expressions (see :mod:metacooler.filters) applied before anything is written, e.g. ["contigs.length >= 5kb"]. A dropped contig keeps none of its bins and every pixel touching it is discarded; a genome left with no contig is left out. Unlike create, genomes.n_bins is available, since the bins come with the inputs.

None
metadata dict | None

Metadata dict stored in the metadata attribute.

None
sort_in_memory bool | None

Only matters when contig_ordering reorders an input's contigs. True re-sorts such inputs in memory, one at a time; False hands the stream to cooler's external sort; None (default) sorts in memory unless the largest such input exceeds 100 M pixels.

None
chunksize int

Pixels read per chunk, and the minimum size of a chunk handed to cooler.

DEFAULT_CHUNKSIZE
qc bool

Tally contact-quality statistics off the pixel stream and store them in metadata["qc"] and the qc_* columns of /genomes and /contigs, as create cool does (source="pixels"). The inter_genome category is 0 by construction.

True
qc_distance_threshold int

Intra-contig short/long split for qc, in bp.

DEFAULT_DISTANCE_THRESHOLD
genomes_table_format str

Which column of genomes_table holds the genome names: a 1-based position or a header name, plus ,noheader for a table without a header line. Defaults to genome_name.

GENOMES_FORMAT_DEFAULT
contigs_table_format str

Which column of contigs_table holds the contig names, in the same grammar. Defaults to contig_name.

CONTIG_METADATA_FORMAT_DEFAULT
keep_partial bool

On a failure while the output is being written, leave <output>.partial behind for inspection, as the default does (logged at ERROR). False removes it instead, whatever stopped the run (KeyboardInterrupt included), and logs that at INFO. The output path itself is never written by a failed run.

True

Raises:

Type Description
ValueError

On an empty or ill-formed input list, inputs of different resolutions, genome or contig names that would not be unique, a malformed metadata table, or an unknown contig_ordering.

FileNotFoundError

If an input or a table does not exist.

FilterError

If a filter is malformed or keeps nothing.

MetaCoolerFormatError

If a genome or contig name is not ASCII.

Source code in src/metacooler/io/collect.py
def collect(
    inputs: Sequence[str | Path],
    output: str | Path,
    names: Sequence[str] | None = None,
    genomes_table: str | Path | None = None,
    contigs_table: str | Path | None = None,
    contig_ordering: CollectOrdering = "cool",
    assembly_name: str | None = None,
    filters: Sequence[str] | None = None,
    metadata: dict | None = None,
    sort_in_memory: bool | None = None,
    chunksize: int = DEFAULT_CHUNKSIZE,
    qc: bool = True,
    qc_distance_threshold: int = DEFAULT_DISTANCE_THRESHOLD,
    genomes_table_format: str = GENOMES_FORMAT_DEFAULT,
    contigs_table_format: str = CONTIG_METADATA_FORMAT_DEFAULT,
    keep_partial: bool = True,
) -> None:
    """Collect single-genome cool files into one metacool file, one genome each.

    Every input becomes one genome of the output, named after its file stem
    (``E_coli.cool`` → ``E_coli``) or by *names*.  Every contig is renamed
    ``<genome>__<contig>`` (see :func:`collected_contig_name`), so contig names
    shared between assemblies cannot collide.  Bins and pixels are carried
    over unchanged; contacts *between* two inputs were never observed, so the
    output's inter-genome blocks are empty.

    Args:
        inputs: Cool files, or cooler URIs such as
            ``file.mcool::resolutions/1000``.  All must share one bin size, or
            all have variable (e.g. restriction-fragment) bins.  Metacool files
            are refused: each input must be a single genome.
        output: Destination ``.metacool`` path.  Written to a ``.partial``
            sibling first and moved into place once it validates.
        names: Genome name for each input, in order.  Defaults to the file
            stems, which must then be unique.
        genomes_table: Optional genomes metadata TSV.  Its genome-name column
            (see *genomes_table_format*) is matched against the genome names;
            every other column is stored in ``/genomes``.  A mismatch in either
            direction is reported as a warning.
        contigs_table: Optional per-contig metadata TSV.  Its contig-name
            column (see *contigs_table_format*) names a contig as in its input
            cool or as collected; every other column is stored in ``/contigs``.
            An optional ``genome_name`` column says which genome each row
            describes — it disambiguates names several inputs share, and is
            checked, never used to move a contig to another genome.
        contig_ordering: ``"cool"`` (default) keeps every input's own contig
            order; ``"genome-then-length"`` puts each genome's longest contig
            first; ``"genome-then-alpha"`` sorts each genome's contigs by name.
            Genomes are always laid out in ``genome_id`` (alphabetical) order.
            ``"cool"`` never needs a pixel sort.
        assembly_name: Stored in the ``genome-assembly`` attribute
            (``"unknown"`` when not given).
        filters: ``--filter`` expressions (see :mod:`metacooler.filters`)
            applied before anything is written, e.g.
            ``["contigs.length >= 5kb"]``.  A dropped contig keeps none of its
            bins and every pixel touching it is discarded; a genome left with
            no contig is left out.  Unlike ``create``, ``genomes.n_bins`` is
            available, since the bins come with the inputs.
        metadata: Metadata dict stored in the ``metadata`` attribute.
        sort_in_memory: Only matters when *contig_ordering* reorders an
            input's contigs.  ``True`` re-sorts such inputs in memory, one at a
            time; ``False`` hands the stream to cooler's external sort;
            ``None`` (default) sorts in memory unless the largest such input
            exceeds 100 M pixels.
        chunksize: Pixels read per chunk, and the minimum size of a chunk
            handed to cooler.
        qc: Tally contact-quality statistics off the pixel stream and store
            them in ``metadata["qc"]`` and the ``qc_*`` columns of ``/genomes``
            and ``/contigs``, as ``create cool`` does (``source="pixels"``).
            The ``inter_genome`` category is 0 by construction.
        qc_distance_threshold: Intra-contig short/long split for *qc*, in bp.
        genomes_table_format: Which column of *genomes_table* holds the genome
            names: a 1-based position or a header name, plus ``,noheader`` for
            a table without a header line.  Defaults to ``genome_name``.
        contigs_table_format: Which column of *contigs_table* holds the contig
            names, in the same grammar.  Defaults to ``contig_name``.
        keep_partial: On a failure while the output is being written, leave
            ``<output>.partial`` behind for inspection, as the default does
            (logged at ERROR).  ``False`` removes it instead, whatever stopped
            the run (``KeyboardInterrupt`` included), and logs that at INFO.
            The output path itself is never written by a failed run.

    Raises:
        ValueError: On an empty or ill-formed input list, inputs of different
            resolutions, genome or contig names that would not be unique, a
            malformed metadata table, or an unknown *contig_ordering*.
        FileNotFoundError: If an input or a table does not exist.
        metacooler.filters.FilterError: If a filter is malformed or keeps
            nothing.
        MetaCoolerFormatError: If a genome or contig name is not ASCII.
    """
    timer = time.perf_counter()
    # A lone path is one input, not a sequence of one-character paths.
    if isinstance(inputs, str | Path):
        inputs = [inputs]
    if isinstance(names, str):
        # Iterating it would silently name two inputs 'A' and 'B' from "AB".
        raise TypeError(
            "names must be a sequence of genome names, one per input, not a "
            f"single string ({names!r})."
        )
    sources = [str(src) for src in inputs]
    output = Path(output)
    if not sources:
        raise ValueError("collect needs at least one input cool file.")
    if contig_ordering not in COLLECT_ORDERINGS:
        raise ValueError(
            f"Unknown contig ordering {contig_ordering!r} for collect; expected "
            f"one of {', '.join(COLLECT_ORDERINGS)}."
        )
    if chunksize < 1:
        raise ValueError(f"chunksize must be a positive integer, got {chunksize}.")
    check_output_not_an_input(output, sources)
    read = [*sources, genomes_table, contigs_table]
    _refuse_staging_an_input(output, read)
    genome_names = _genome_names(sources, names)

    # --- Read every input's bins before a single pixel ---
    inputs_ = [
        _read_input(src, genome)
        for src, genome in zip(sources, genome_names, strict=True)
    ]
    binsize = _check_resolutions(inputs_)
    count_dtype = np.result_type(*(inp.count_dtype for inp in inputs_))
    for inp in inputs_:
        logger.debug(
            "Genome %r <- %s: %d contig(s), %d bin(s), %d pixel(s)",
            inp.genome,
            inp.source,
            len(inp.chrom_names),
            inp.chrom.size,
            inp.nnz,
        )
    weighted = [inp.source for inp in inputs_ if inp.has_weight]
    if weighted:
        logger.info(
            "Balancing weights (bins/weight) are not carried over from %d "
            "input(s) (%s); re-balance the collected map with 'cooler balance' "
            "if you need them.",
            len(weighted),
            _examples(weighted),
        )

    # --- Name every contig, then read the optional tables against the names ---
    catalog = _contig_catalog(inputs_)
    contig_lengths: dict[str, int] = dict(
        zip(catalog["name"], catalog["length"].tolist(), strict=True)
    )
    contig_genome: dict[str, str] = dict(
        zip(catalog["name"], catalog["genome"], strict=True)
    )
    extra_genomes_df = _load_extra_genomes_table(genomes_table, genomes_table_format)
    if extra_genomes_df is not None:
        _report_genome_divergence(genome_names, extra_genomes_df)
    contig_df = (
        _load_contig_metadata(contigs_table, catalog, contigs_table_format)
        if contigs_table is not None
        else None
    )

    # --- Apply --filter predicates before anything is written ---
    contig_lengths, _ = _apply_create_filters(
        contig_lengths,
        contig_genome,
        contig_df,
        extra_genomes_df,
        filters,
        genome_n_bins=catalog.groupby("genome")["n_bins"].sum().to_dict(),
        report_divergence=False,
        dropped_effect="keep none of their bins, and every pixel touching one "
        "is discarded",
    )

    # --- Order the contigs and build the /genomes and /contigs tables ---
    contig_order = _contig_order(
        inputs_, contig_lengths, contig_genome, contig_ordering
    )
    genomes_df = _build_genomes_table(
        contig_order,
        contig_lengths,
        contig_genome,
        extra_genomes_df,
        report_divergence=False,
    )
    contigs_df = _build_contigs_table(
        contig_order, contig_lengths, contig_genome, genomes_df, contig_df
    )
    # Before the pixels are streamed: a name cooler cannot read costs nothing
    # here and a whole pass over every input if it is only caught at write time.
    check_ascii_names(genomes_df, contigs_df)

    # --- Lay out the bins, one block per surviving genome ---
    by_genome = {inp.genome: inp for inp in inputs_}
    contig_id = {name: i for i, name in enumerate(contig_order)}
    bins_df, blocks = _layout_bins(
        [by_genome[g] for g in genomes_df["name"]], contig_id
    )
    chrom_offset, genome_offset = _compute_all_offsets(bins_df, contigs_df, genomes_df)
    del catalog, contig_df, extra_genomes_df, contig_genome, contig_lengths
    logger.info(
        "Collecting %d genome(s) from %d input(s): %d contig(s), %d bin(s).",
        len(genomes_df),
        len(inputs_),
        len(contigs_df),
        len(bins_df),
    )

    # --- Stream the pixels, block after block ---
    sort = _choose_sort(blocks, sort_in_memory)
    ordered = sort != "external"

    def _pixels() -> Iterator[pd.DataFrame]:
        for block in blocks:
            if sort == "memory" and block.needs_sort:
                yield from _sorted_block_pixels(block, count_dtype, chunksize)
            else:
                yield from _block_pixels(block, count_dtype, chunksize)

    pixel_source: Iterable[pd.DataFrame] = _coalesce(_pixels(), chunksize)
    qc_tally = (
        ContactQC(
            len(genomes_df),
            qc_distance_threshold,
            source="pixels",
            n_contigs=len(contigs_df),
        )
        if qc
        else None
    )
    if qc_tally is not None:
        pixel_source = _tally_pixel_qc(
            pixel_source, qc_tally, bin_maps(bins_df, contigs_df)
        )
    pixel_source, was_empty = _ensure_nonempty_pixel_stream(pixel_source)
    ordered = ordered or was_empty
    with _staged_output(
        output,
        keep_partial=keep_partial,
        inputs=read,
    ) as staged:
        write_cooler(
            staged,
            bins_df,
            pixel_source,
            binsize=binsize,
            ordered=ordered,
            symmetric_upper=True,
            assembly=assembly_name,
            metadata=metadata or {},
            dtypes={"count": count_dtype},
        )

        logger.info("Post-processing cooler file to add metacool structure")
        _finalize_metacool_file(
            staged,
            genomes_df,
            contigs_df,
            chrom_offset,
            genome_offset,
            contig_ordering=contig_ordering,
            assembly_name=assembly_name,
            binsize=binsize,
        )

        # Complete only now: the tally rides on the stream write_cooler consumed.
        if qc_tally is not None:
            store_qc(
                staged,
                qc_tally.summary(binsize=binsize),
                qc_tally.genome_table(genomes_df["name"].to_numpy()),
                qc_tally.contig_table(contigs_df["name"].to_numpy()),
            )

        logger.info("Validating collected metacool file")
        validate(staged)

    with h5py.File(output, "r") as f:
        nnz = int(f.attrs["nnz"])
        total = f.attrs["sum"]
    logger.info(
        "Collected %d genome(s), %d contig(s), %d bin(s). NNZ=%d, sum=%s.",
        len(genomes_df),
        len(contigs_df),
        len(bins_df),
        nnz,
        total,
    )
    logger.info(
        "Created metacool file: %s (%s elapsed)", output, _total_time_str(timer)
    )

collected_contig_name

collected_contig_name(genome: str, contig: str) -> str

Return the name contig contig of genome genome is collected under.

The name is <genome>__<contig>, unless contig already starts with exactly that prefix: a cool extracted from an earlier collection carries its genome prefix already, and collecting it again must not stack a second one (E_coli__E_coli__chr1).

Parameters:

Name Type Description Default
genome str

Genome name.

required
contig str

Contig name as it appears in the input cool.

required

Returns:

Type Description
str

The collected contig name.

Source code in src/metacooler/io/collect.py
def collected_contig_name(genome: str, contig: str) -> str:
    """Return the name contig *contig* of genome *genome* is collected under.

    The name is ``<genome>__<contig>``, unless *contig* already starts with
    exactly that prefix: a cool extracted from an earlier collection carries
    its genome prefix already, and collecting it again must not stack a second
    one (``E_coli__E_coli__chr1``).

    Args:
        genome: Genome name.
        contig: Contig name as it appears in the input cool.

    Returns:
        The collected contig name.
    """
    prefix = f"{genome}{CONTIG_NAME_SEPARATOR}"
    return contig if contig.startswith(prefix) else prefix + contig

Reassigning contigs to genomes

reassign(
    uri: str | Path,
    contig_genome_map: str | Path | dict[str, str],
    output: str | Path | None = None,
    *,
    contig_table_format: str = CONTIGS_FORMAT_DEFAULT,
    genomes_table: str | Path | None = None,
    genomes_table_format: str = GENOMES_FORMAT_DEFAULT,
    missing: Literal["keep", "unbinned"] = "keep",
    layout: Layout = "auto",
    inplace: bool = False,
    metadata: dict | None = None,
    qc: bool = True,
    chunksize: int = DEFAULT_CHUNKSIZE,
    force: bool = False,
    keep_partial: bool = True,
) -> None

Reassign the contigs of a metacool file to different genomes.

The contact matrix is never recomputed: contigs, their lengths and their bins are exactly those of the source file, and every pixel keeps its count. What changes is which genome each contig belongs to, and everything derived from that — the /genomes table, the genome_id column, and (on the regroup path) the order the contigs are stored in.

Parameters:

Name Type Description Default
uri str | Path

Source .metacool path or file.mmetacool::entries/name URI.

required
contig_genome_map str | Path | dict[str, str]

Reassignment table (path), or a ready-made {contig: genome} mapping. It need not cover every contig — see missing.

required
output str | Path | None

Destination .metacool path. Required unless inplace.

None
contig_table_format str

Which columns of the reassignment table hold the contig and the genome names, as accepted by :func:metacooler.io.contigs.parse_contigs_table: CONTIG,GENOME[,noheader], each a 1-based position or a header name, or an alias. Extra columns of the table are ignored; only the two name columns are read.

CONTIGS_FORMAT_DEFAULT
genomes_table str | Path | None

Optional --genomes metadata table for the new genomes. When given it replaces the metadata columns carried over from the source /genomes table rather than merging with them.

None
genomes_table_format str

Which column of genomes_table holds the genome names: a 1-based position or a header name, plus ,noheader for a table without a header line. Defaults to genome_name.

GENOMES_FORMAT_DEFAULT
missing Literal['keep', 'unbinned']

What to do with a contig the table does not mention. "keep" (default) leaves its current genome, which makes a short table an incremental edit; "unbinned" moves it to __unbinned__, which makes the table authoritative — the right choice when it is the output of a fresh binning run. Neither drops a contig: removing contigs (and their pixels) is what :func:metacooler.io.extract.extract is for.

'keep'
layout Layout

How to lay the output's contigs out. "auto" (default) re-groups a file whose contig-ordering is genome-grouped and leaves any other file's order alone; "keep" never re-orders; "genome-then-alpha"/"genome-then-length" force a re-grouping with that strategy. Re-ordering costs a full pixel rewrite and is skipped automatically when the strategy reproduces the order the file already has.

'auto'
inplace bool

Modify uri itself instead of writing a new file. Only possible when no re-ordering is required, since permuting the bins means rewriting the pixel table.

False
metadata dict | None

Extra keys to merge into the output's metadata attribute. A commands list is appended to the carried provenance rather than replacing it.

None
qc bool

Keep the contact-quality block when the source carries one, for the new genomes: the per-contig intra-contig counts are carried (so a create pairs split measured on read positions survives) and the inter-contig categories are re-measured from the pixels. A source without per-contig counts (written by metacooler 0.2.0 or earlier) has its whole block recomputed from the pixels. With False the block is dropped and not replaced.

True
chunksize int

Pixels read per chunk: remapped on the re-ordering path, and classified when the QC block is stored.

DEFAULT_CHUNKSIZE
force bool

Replace output if it exists. The existing file is only replaced once the new one has been written and validated, so a failed run leaves it untouched.

False
keep_partial bool

On a failure while the output is being written, leave <output>.partial behind for inspection, as the default does (logged at ERROR). False removes it instead, whatever stopped the run (KeyboardInterrupt included), and logs that at INFO. The output path itself is never written by a failed run. Unused with inplace, which stages nothing.

True

Raises:

Type Description
ValueError

If the output arguments are inconsistent, if output is one of the inputs, if inplace is asked for a reassignment that requires re-ordering, if re-ordering is asked of a map whose storage mode is not symmetric-upper, or if the table names no contig of this file.

FileExistsError

If output exists and force is not set.

Source code in src/metacooler/io/reassign.py
def reassign(
    uri: str | Path,
    contig_genome_map: str | Path | dict[str, str],
    output: str | Path | None = None,
    *,
    contig_table_format: str = CONTIGS_FORMAT_DEFAULT,
    genomes_table: str | Path | None = None,
    genomes_table_format: str = GENOMES_FORMAT_DEFAULT,
    missing: Literal["keep", "unbinned"] = "keep",
    layout: Layout = "auto",
    inplace: bool = False,
    metadata: dict | None = None,
    qc: bool = True,
    chunksize: int = DEFAULT_CHUNKSIZE,
    force: bool = False,
    keep_partial: bool = True,
) -> None:
    """Reassign the contigs of a metacool file to different genomes.

    The contact matrix is never recomputed: contigs, their lengths and their
    bins are exactly those of the source file, and every pixel keeps its count.
    What changes is which genome each contig belongs to, and everything derived
    from that — the ``/genomes`` table, the ``genome_id`` column, and (on the
    regroup path) the order the contigs are stored in.

    Args:
        uri: Source ``.metacool`` path or ``file.mmetacool::entries/name`` URI.
        contig_genome_map: Reassignment table (path), or a ready-made
            ``{contig: genome}`` mapping.  It need not cover every contig — see
            *missing*.
        output: Destination ``.metacool`` path.  Required unless *inplace*.
        contig_table_format: Which columns of the reassignment table hold the
            contig and the genome names, as accepted by
            :func:`metacooler.io.contigs.parse_contigs_table`:
            ``CONTIG,GENOME[,noheader]``, each a 1-based position or a header
            name, or an alias.  Extra columns of the table are ignored; only
            the two name columns are read.
        genomes_table: Optional ``--genomes`` metadata table for the *new*
            genomes.  When given it replaces the metadata columns carried over
            from the source ``/genomes`` table rather than merging with them.
        genomes_table_format: Which column of *genomes_table* holds the genome
            names: a 1-based position or a header name, plus ``,noheader`` for
            a table without a header line.  Defaults to ``genome_name``.
        missing: What to do with a contig the table does not mention.
            ``"keep"`` (default) leaves its current genome, which makes a short
            table an incremental edit; ``"unbinned"`` moves it to
            ``__unbinned__``, which makes the table authoritative — the right
            choice when it is the output of a fresh binning run.  Neither drops
            a contig: removing contigs (and their pixels) is what
            :func:`metacooler.io.extract.extract` is for.
        layout: How to lay the output's contigs out.  ``"auto"`` (default)
            re-groups a file whose ``contig-ordering`` is genome-grouped and
            leaves any other file's order alone; ``"keep"`` never re-orders;
            ``"genome-then-alpha"``/``"genome-then-length"`` force a re-grouping
            with that strategy.  Re-ordering costs a full pixel rewrite and is
            skipped automatically when the strategy reproduces the order the
            file already has.
        inplace: Modify *uri* itself instead of writing a new file.  Only
            possible when no re-ordering is required, since permuting the bins
            means rewriting the pixel table.
        metadata: Extra keys to merge into the output's ``metadata`` attribute.
            A ``commands`` list is appended to the carried provenance rather
            than replacing it.
        qc: Keep the contact-quality block when the source carries one, for
            the new genomes: the per-contig intra-contig counts are carried
            (so a ``create pairs`` split measured on read positions survives)
            and the inter-contig categories are re-measured from the pixels.
            A source without per-contig counts (written by metacooler 0.2.0 or
            earlier) has its whole block recomputed from the pixels.  With
            ``False`` the block is dropped and not replaced.
        chunksize: Pixels read per chunk: remapped on the re-ordering path, and
            classified when the QC block is stored.
        force: Replace *output* if it exists.  The existing file is only
            replaced once the new one has been written and validated, so a
            failed run leaves it untouched.
        keep_partial: On a failure while the output is being written, leave
            ``<output>.partial`` behind for inspection, as the default does
            (logged at ERROR).  ``False`` removes it instead, whatever stopped
            the run (``KeyboardInterrupt`` included), and logs that at INFO.
            The output path itself is never written by a failed run.  Unused
            with *inplace*, which stages nothing.

    Raises:
        ValueError: If the output arguments are inconsistent, if *output* is
            one of the inputs, if *inplace* is asked for a reassignment that
            requires re-ordering, if re-ordering is asked of a map whose
            storage mode is not ``symmetric-upper``, or if the table names no
            contig of this file.
        FileExistsError: If *output* exists and *force* is not set.
    """
    from metacooler.api import MetaCooler

    if inplace and output is not None:
        raise ValueError(
            "inplace rewrites the input file, so no output path is expected; "
            "pass one or the other."
        )
    if not inplace and output is None:
        raise ValueError(
            "An output path is required. Pass output=... to write a new file, "
            "or inplace=True to rewrite the input."
        )
    inputs = [
        uri,
        genomes_table,
        None if isinstance(contig_genome_map, dict) else contig_genome_map,
    ]
    if output is not None:
        # Before anything is read: an output (or the partial file it is staged
        # as) that is one of the inputs is refused even under force, and an
        # existing output is left in place for _staged_output to replace once
        # the new file has validated.
        check_output_path(output, force, inputs=inputs, defer_removal=True)
        _refuse_staging_an_input(output, inputs)

    mc = MetaCooler(uri)
    old_contigs = mc.contigs
    old_genomes = mc.genomes
    contig_names = old_contigs["name"].astype(str).tolist()
    genome_names = old_genomes["name"].astype(str).to_numpy()
    before = dict(
        zip(
            contig_names,
            genome_names[old_contigs["genome_id"].to_numpy(np.int64)],
            strict=True,
        )
    )

    source_table: str | Path | dict[str, str] = (
        {str(k): str(v) for k, v in contig_genome_map.items()}
        if isinstance(contig_genome_map, dict)
        else contig_genome_map
    )
    table, _ = _parse_contig_map(source_table, contig_table_format)
    assignment, report = _apply_reassignment(before, table, missing)
    _log_report(report, len(contig_names))

    contig_lengths = {
        name: int(length)
        for name, length in zip(
            contig_names, old_contigs["length"].to_numpy(np.int64), strict=True
        )
    }
    source_ordering = mc.info.get("ordering")
    strategy = _resolve_layout(layout, source_ordering)
    if strategy == "keep":
        contig_order = contig_names
    else:
        contig_order = _build_contig_order(
            contig_lengths, assignment, ordering=cast("ContigOrderStrategy", strategy)
        )
    reorder = contig_order != contig_names
    # The label follows the *strategy*, not the amount of movement: a requested
    # layout that happens to reproduce the existing order still describes the
    # rows correctly, and writing the source's old label there would understate
    # what the file now guarantees.  Only "keep" has to interrogate the rows.
    ordering = (
        _kept_ordering(
            source_ordering, contig_order, contig_lengths, before, assignment
        )
        if strategy == "keep"
        else strategy
    )

    storage_mode = mc.info.get("storage-mode", "symmetric-upper")
    if reorder and storage_mode != "symmetric-upper":
        # Remapping folds every pixel into the upper triangle, which is only
        # right for a symmetric matrix -- and square storage is chosen precisely
        # when one is not guaranteed, so a pixel and its mirror would be summed.
        # Relabelling never touches a pixel, so it stays available.
        raise ValueError(
            f"{uri} uses storage-mode {storage_mode!r}, and laying its contigs "
            f"out as {strategy!r} would fold every pixel into the upper "
            "triangle, summing each contact with its mirror. Pass layout='keep' "
            "to relabel without moving any pixel."
        )
    if reorder and inplace:
        raise ValueError(
            f"inplace cannot be used here: laying the contigs out as {strategy!r} "
            "moves them, which means permuting every bin and rewriting the whole "
            "pixel table. Write a new file with output=..., or pass "
            "layout='keep' to relabel without re-ordering (the genomes' bins "
            "then stop being contiguous)."
        )

    genomes = _build_genomes_table(
        contig_order=contig_order,
        contig_lengths=contig_lengths,
        contig_genome_map=assignment,
        extra_genomes_df=(
            _load_extra_genomes_table(genomes_table, genomes_table_format)
            if genomes_table is not None
            else _carry_genome_metadata(
                old_genomes,
                set(assignment.values()),
                {before[c] for c in report.moved}
                | {assignment[c] for c in report.moved},
            )
        ),
    )
    contigs = _build_contigs_table(
        contig_order=contig_order,
        contig_lengths=contig_lengths,
        contig_genome_map=assignment,
        genomes=genomes,
        contig_df=_carry_contig_metadata(old_contigs, assignment),
    )
    genomes = strip_qc_columns(genomes)
    contigs = strip_contig_qc_columns(contigs)
    if not reorder:
        # The re-ordering path gets n_bins from _compute_all_offsets, which needs
        # the bins table; here the bins have not moved, so the per-contig counts
        # already on disk answer the same question without reading them.
        _insert_n_bins(mc, genomes, contigs)

    source_metadata = mc.metadata
    output_metadata, had_qc = _build_metadata(
        source_metadata,
        metadata,
        source_uri=str(uri),
        table=str(contig_genome_map) if not isinstance(contig_genome_map, dict) else "",
        layout=strategy,
        n_moved=len(report.moved),
    )
    block = source_metadata.get(QC_METADATA_KEY)
    carried = _carried_contig_qc(block, old_contigs) if had_qc and qc else None
    if not report.moved:
        _log_nothing_moved(
            genomes_table=genomes_table is not None,
            reorder=reorder,
            strategy=strategy,
            source_ordering=source_ordering,
            ordering=ordering,
            qc_block=(
                None
                if not had_qc or carried is not None
                else "recomputed"
                if qc
                else "dropped"
            ),
            metadata_keys=[k for k in (metadata or {}) if k != "commands"],
        )
    if had_qc and qc:
        # Measured before anything is written, and written with the rest of
        # the map: a run stopped meanwhile leaves the source's block intact.
        new_qc = _measure_qc(mc, genomes, contigs, block, carried, chunksize)
        output_metadata[QC_METADATA_KEY] = new_qc.summary
        genomes = _with_columns(genomes, new_qc.per_genome, QC_GENOME_COLUMNS)
        contigs = _with_columns(contigs, new_qc.per_contig, QC_CONTIG_COLUMNS)

    if inplace:
        _relabel_in_place(mc, genomes, contigs, ordering, output_metadata)
        target_uri = mc.uri
        validate(target_uri)
    else:
        assert output is not None  # guaranteed by the argument check above
        target = Path(output)
        with _staged_output(target, keep_partial=keep_partial, inputs=inputs) as staged:
            if reorder:
                logger.info(
                    "Re-ordering %d contigs into %r layout: the bins move with "
                    "them, so the %s pixels are remapped and re-sorted.",
                    len(contig_order),
                    strategy,
                    f"{int(mc.info['nnz']):,}",
                )
                _rewrite_regrouped(
                    mc,
                    staged,
                    genomes,
                    contigs,
                    contig_order,
                    ordering,
                    output_metadata,
                    chunksize,
                )
            else:
                _materialise_copy(mc, staged)
                _write_relabelled(
                    staged,
                    "/",
                    genomes,
                    contigs,
                    _genome_offset(contigs, len(genomes)),
                    ordering,
                    output_metadata,
                )
            validate(staged)
        target_uri = str(target)

    if had_qc and not qc:
        logger.warning(
            "The source's QC block describes the old genome assignment and "
            "was dropped; run 'metacooler qc --store %s' to measure the new "
            "one.",
            target_uri,
        )

    logger.info(
        "Reassigned %d contig(s) across %d genome(s) → %s",
        len(report.moved),
        len(genomes),
        target_uri,
    )

Annotating genomes and contigs

annotate(
    uri: str | Path,
    output: str | Path | None = None,
    *,
    genomes_table: str | Path | None = None,
    contigs_table: str | Path | None = None,
    genomes_table_format: str = GENOMES_FORMAT_DEFAULT,
    contigs_table_format: str = CONTIG_METADATA_FORMAT_DEFAULT,
    columns: Sequence[str] | None = None,
    drop: Sequence[str] | None = None,
    replace: bool = False,
    missing: Missing = "keep",
    unmatched: Unmatched = "warn",
    inplace: bool = False,
    metadata: dict | None = None,
    force: bool = False,
    keep_partial: bool = True,
) -> None

Add, replace or drop metadata columns of /genomes and /contigs.

Parameters:

Name Type Description Default
uri str | Path

A .metacool path, an .mmetacool entry URI (file.mmetacool::entries/<name>), or a bare .mmetacool path, which annotates every entry of the collection.

required
output str | Path | None

Where to write the annotated copy. A whole file in gives a whole file out (a collection stays a collection); an entry URI gives a standalone .metacool. Required unless inplace.

None
genomes_table str | Path | None

Genomes metadata TSV. Its genome-name column (see genomes_table_format) is matched against /genomes/name; every other column is metadata.

None
contigs_table str | Path | None

Per-contig metadata TSV, as collect reads it: its contig-name column (see contigs_table_format) names a contig as the map stores it (or, in a collected map, as in its input cool); an optional genome_name column is checked, never applied; every other column is metadata.

None
genomes_table_format str

Which column of genomes_table holds the genome names: a 1-based position or a header name, plus ,noheader.

GENOMES_FORMAT_DEFAULT
contigs_table_format str

Which column of contigs_table holds the contig names, in the same grammar.

CONTIG_METADATA_FORMAT_DEFAULT
columns Sequence[str] | None

Take only these metadata columns, as genomes.<name> / contigs.<name> (named as in the tables). Default: all of them.

None
drop Sequence[str] | None

Remove these columns, as genomes.<name> / contigs.<name>.

None
replace bool

Allow changing the values of a column the map already has. A column given the values it already holds is never a change.

False
missing Missing

For a replaced column, what the rows the table does not list get: "keep" (their current value: the table is a patch) or "na" (an empty value: the table is authoritative). New columns always leave unlisted rows empty.

'keep'
unmatched Unmatched

What to do with the rows of a table that name no genome (contig) of a map: "warn" (default) ignores them with a WARNING, "ignore" without one (DEBUG only), "error" refuses the tables before anything is written. A table that matches nothing at all is refused whatever this says.

'warn'
inplace bool

Edit uri itself instead of writing output.

False
metadata dict | None

Keys merged into each edited map's metadata attribute; a commands list is appended to the provenance already there.

None
force bool

Replace output if it exists (only once the new file validates).

False
keep_partial bool

On a failure while the output is being written, leave <output>.partial behind for inspection, as the default does (logged at ERROR). False removes it instead, whatever stopped the run (KeyboardInterrupt included), and logs that at INFO. The output path itself is never written by a failed run. Unused with inplace, which stages nothing.

True

Raises:

Type Description
ValueError

On inconsistent arguments, a table that matches nothing (or, under unmatched="error", a row that does), a column the tables would change without replace, a derived column in drop, or a contig table contradicting the genome assignment.

MetaCoolerFormatError

If a /contigs column would hold non-ASCII text (cooler could no longer open the map).

FileNotFoundError

If the map or a table does not exist.

Source code in src/metacooler/io/annotate.py
def annotate(
    uri: str | Path,
    output: str | Path | None = None,
    *,
    genomes_table: str | Path | None = None,
    contigs_table: str | Path | None = None,
    genomes_table_format: str = GENOMES_FORMAT_DEFAULT,
    contigs_table_format: str = CONTIG_METADATA_FORMAT_DEFAULT,
    columns: Sequence[str] | None = None,
    drop: Sequence[str] | None = None,
    replace: bool = False,
    missing: Missing = "keep",
    unmatched: Unmatched = "warn",
    inplace: bool = False,
    metadata: dict | None = None,
    force: bool = False,
    keep_partial: bool = True,
) -> None:
    """Add, replace or drop metadata columns of ``/genomes`` and ``/contigs``.

    Args:
        uri: A ``.metacool`` path, an ``.mmetacool`` entry URI
            (``file.mmetacool::entries/<name>``), or a bare ``.mmetacool``
            path, which annotates every entry of the collection.
        output: Where to write the annotated copy.  A whole file in gives a
            whole file out (a collection stays a collection); an entry URI
            gives a standalone ``.metacool``.  Required unless *inplace*.
        genomes_table: Genomes metadata TSV.  Its genome-name column (see
            *genomes_table_format*) is matched against ``/genomes/name``; every
            other column is metadata.
        contigs_table: Per-contig metadata TSV, as ``collect`` reads it: its
            contig-name column (see *contigs_table_format*) names a contig as
            the map stores it (or, in a collected map, as in its input cool);
            an optional ``genome_name`` column is checked, never applied;
            every other column is metadata.
        genomes_table_format: Which column of *genomes_table* holds the genome
            names: a 1-based position or a header name, plus ``,noheader``.
        contigs_table_format: Which column of *contigs_table* holds the contig
            names, in the same grammar.
        columns: Take only these metadata columns, as ``genomes.<name>`` /
            ``contigs.<name>`` (named as in the tables).  Default: all of them.
        drop: Remove these columns, as ``genomes.<name>`` / ``contigs.<name>``.
        replace: Allow changing the values of a column the map already has.
            A column given the values it already holds is never a change.
        missing: For a replaced column, what the rows the table does not list
            get: ``"keep"`` (their current value: the table is a patch) or
            ``"na"`` (an empty value: the table is authoritative).  New
            columns always leave unlisted rows empty.
        unmatched: What to do with the rows of a table that name no genome
            (contig) of a map: ``"warn"`` (default) ignores them with a
            WARNING, ``"ignore"`` without one (DEBUG only), ``"error"``
            refuses the tables before anything is written.  A table that
            matches nothing at all is refused whatever this says.
        inplace: Edit *uri* itself instead of writing *output*.
        metadata: Keys merged into each edited map's ``metadata`` attribute; a
            ``commands`` list is appended to the provenance already there.
        force: Replace *output* if it exists (only once the new file
            validates).
        keep_partial: On a failure while the output is being written, leave
            ``<output>.partial`` behind for inspection, as the default does
            (logged at ERROR).  ``False`` removes it instead, whatever stopped
            the run (``KeyboardInterrupt`` included), and logs that at INFO.
            The output path itself is never written by a failed run.  Unused
            with *inplace*, which stages nothing.

    Raises:
        ValueError: On inconsistent arguments, a table that matches nothing (or,
            under ``unmatched="error"``, a row that does), a column the tables
            would change without *replace*, a derived column in *drop*, or a
            contig table contradicting the genome assignment.
        MetaCoolerFormatError: If a ``/contigs`` column would hold non-ASCII
            text (cooler could no longer open the map).
        FileNotFoundError: If the map or a table does not exist.
    """
    from metacooler.api import MetaCooler

    if inplace and output is not None:
        raise ValueError(
            "inplace edits the input, so no output path is expected; pass one "
            "or the other."
        )
    if not inplace and output is None:
        raise ValueError(
            "An output path is required. Pass output=... to write an annotated "
            "copy, or inplace=True to edit the input."
        )
    if missing not in ("keep", "na"):
        raise ValueError(f"missing must be 'keep' or 'na', got {missing!r}.")
    if unmatched not in UNMATCHED_CHOICES:
        raise ValueError(
            f"unmatched must be 'warn', 'ignore' or 'error', got {unmatched!r}."
        )
    wanted = _qualified_names(columns, "columns")
    dropping = _qualified_names(drop, "drop")
    if genomes_table is None and contigs_table is None and not dropping:
        raise ValueError(
            "Nothing to do: give a genomes table, a contigs table, or columns to drop."
        )
    _check_drops(dropping, wanted)
    for table, given in (("genomes", genomes_table), ("contigs", contigs_table)):
        if given is None and wanted.get(table):
            raise ValueError(
                f"columns names {table} column(s) "
                f"({', '.join(wanted[table])}) but no {table} table was given."
            )
    inputs = [uri, genomes_table, contigs_table]
    if output is not None:
        from metacooler.io.create import _refuse_staging_an_input

        check_output_path(output, force, inputs=inputs, defer_removal=True)
        _refuse_staging_an_input(output, inputs)

    # --- Read the tables once; they do not depend on the map ---
    genomes_extra = (
        _read_genomes(genomes_table, genomes_table_format, wanted)
        if genomes_table is not None
        else None
    )
    contig_table = (
        _read_contigs(contigs_table, contigs_table_format, wanted)
        if contigs_table is not None
        else None
    )

    # --- Plan every map before writing any ---
    file_path, group_path = parse_uri(str(uri))
    groups = _map_groups(file_path, group_path)
    edits = []
    # Rows refused under unmatched="error", raised once every map is planned:
    # a table that matches nothing at all gets its own, clearer refusal first.
    unmatched_rows: list[str] = []
    for group in groups:
        map_uri = f"{file_path}::{group}"
        if len(groups) > 1:
            logger.info("Annotating entry %s", group)
        edits.append(
            _plan_map(
                MetaCooler(map_uri),
                group,
                genomes_extra,
                contig_table,
                contigs_table,
                dropping,
                replace=replace,
                missing=missing,
                unmatched=unmatched,
                unmatched_rows=unmatched_rows,
            )
        )
    _check_plans(edits, genomes_table, contigs_table, dropping, unmatched_rows)

    history = {
        "genomes_table": None if genomes_table is None else str(genomes_table),
        "contigs_table": None if contigs_table is None else str(contigs_table),
        "replace": replace,
        "missing": missing,
    }
    if inplace:
        for edit in edits:
            _write_map(file_path, edit.group, edit, metadata, history)
            if edit.changed:
                validate(edit.uri)
        return

    from metacooler.io.create import _staged_output

    assert output is not None  # guaranteed by the argument checks above
    target = Path(output)
    with _staged_output(target, keep_partial=keep_partial, inputs=inputs) as staged:
        if group_path.strip("/") and len(groups) == 1:
            # One entry out of a collection: a standalone map, as reassign does.
            from metacooler.io.reassign import _materialise_copy

            _materialise_copy(MetaCooler(f"{file_path}::{group_path}"), staged)
            moved = [dataclasses.replace(edits[0], group="/")]
        else:
            shutil.copy2(file_path, staged)
            moved = edits
        for edit in moved:
            _write_map(staged, edit.group, edit, metadata, history, always=True)
            validate(f"{staged}::{edit.group}")
    logger.info("Annotated map written to %s", target)

Merging metacool files

merge(inputs: Iterable[str | Path], output: str | Path) -> None

Merge multiple .metacool files with identical axes into one.

All input files must describe the same contigs and genomes -- names, lengths, contig-to-genome assignment, bins per genome -- with the same bins and the same contig-ordering attribute. Their metadata columns (completeness, coverage, ...) may differ: the output keeps the first input's, and each difference is reported as a warning. Pixels from all inputs are concatenated, re-sorted externally by cooler.merge_coolers(), and saved to output. Metacooler-specific groups (genomes, contigs, genome_offset, format attributes) are then copied from the first input file.

Parameters:

Name Type Description Default
inputs Iterable[str | Path]

Iterable of paths to the input .metacool files, or file.mmetacool::entries/<name> URIs. At least two are required.

required
output str | Path

Destination path for the merged .metacool file.

required

Raises:

Type Description
ValueError

If fewer than two inputs are provided, if the contigs, genomes or bins tables differ across files, if the resolutions differ, or if the contig-ordering attribute is inconsistent.

MetaCoolerFormatError

If any input file fails the metacooler format validation check.

Source code in src/metacooler/io/merge.py
def merge(
    inputs: Iterable[str | Path],
    output: str | Path,
) -> None:
    """Merge multiple ``.metacool`` files with identical axes into one.

    All input files must describe the same contigs and genomes -- names,
    lengths, contig-to-genome assignment, bins per genome -- with the same
    bins and the same ``contig-ordering`` attribute.  Their metadata columns
    (completeness, coverage, ...) may differ: the output keeps the first
    input's, and each difference is reported as a warning.  Pixels from all
    inputs are concatenated, re-sorted externally by ``cooler.merge_coolers()``,
    and saved to *output*.  Metacooler-specific groups (``genomes``,
    ``contigs``, ``genome_offset``, format attributes) are then copied from the
    first input file.

    Args:
        inputs: Iterable of paths to the input ``.metacool`` files, or
            ``file.mmetacool::entries/<name>`` URIs.  At least two are
            required.
        output: Destination path for the merged ``.metacool`` file.

    Raises:
        ValueError: If fewer than two inputs are provided, if the
            ``contigs``, ``genomes`` or ``bins`` tables differ across files,
            if the resolutions differ, or if the ``contig-ordering``
            attribute is inconsistent.
        MetaCoolerFormatError: If any input file fails the metacooler
            format validation check.
    """
    inputs = [str(p) for p in inputs]
    if len(inputs) < 2:
        raise ValueError("At least two input files required for merge")
    _refuse_repeated_inputs(inputs)
    # cooler writes the output in "w" mode; refuse to truncate an input with it.
    check_output_path(output, force=True, inputs=inputs, defer_removal=True)

    # Validate and compare axes
    first = inputs[0]
    validate(first)

    with _open_uri(first) as f0:
        contigs_hash = _hash_group(f0, "contigs")
        genomes_hash = _hash_group(f0, "genomes")
        metadata_digests = _metadata_digests(f0)
        # ``contig-ordering`` may be absent or explicitly null; normalise both
        # sides of the comparison so a missing attribute does not later become
        # ``f.attrs[...] = None``, which h5py rejects with a raw TypeError.
        ordering = f0.attrs.get("contig-ordering") or "genome-then-alpha"
        n_bins, bins_hash = _bins_signature(f0)
        binsize = _binsize(f0)

    for inp in inputs[1:]:
        validate(inp)
        with _open_uri(inp) as f:
            # Order matters: report the ROOT cause, not a downstream symptom.
            # ``/genomes`` carries ``n_bins``, so its hash changes whenever the
            # resolution does — checking it first made every resolution
            # mismatch report as "Genomes table mismatch", which sends the user
            # looking at the wrong table. Contigs (assembly identity) first,
            # then resolution, then the bins table, and only then genomes.
            if _hash_group(f, "contigs") != contigs_hash:
                raise ValueError(f"Contigs table mismatch: {first} vs {inp}")
            # Merging files binned at different resolutions would concatenate
            # pixels that address unrelated bin IDs, producing a valid-looking
            # but meaningless map, so refuse before any output is written.
            other_binsize = _binsize(f)
            if other_binsize != binsize:
                raise ValueError(
                    f"Resolution mismatch: {first} has bin-size {binsize!r} but "
                    f"{inp} has bin-size {other_binsize!r}. All inputs must be "
                    "binned identically."
                )
            other_n_bins, other_bins_hash = _bins_signature(f)
            if other_n_bins != n_bins:
                raise ValueError(
                    f"Bins table mismatch: {first} has {n_bins} bins but {inp} "
                    f"has {other_n_bins}."
                )
            if other_bins_hash != bins_hash:
                raise ValueError(
                    f"Bins table mismatch: {first} and {inp} have the same number "
                    "of bins but different bin coordinates."
                )
            if _hash_group(f, "genomes") != genomes_hash:
                raise ValueError(f"Genomes table mismatch: {first} vs {inp}")
            if (f.attrs.get("contig-ordering") or "genome-then-alpha") != ordering:
                raise ValueError(f"contig-ordering mismatch: {first} vs {inp}")
            differences = _metadata_differences(metadata_digests, _metadata_digests(f))
            if differences:
                logger.warning(
                    "%s has other metadata columns than %s: %s. The merged map "
                    "keeps those of %s.",
                    inp,
                    first,
                    ", ".join(differences),
                    first,
                )

    # Use cooler's merge logic on the pixel tables.  ``dtypes`` is mandatory:
    # without it the summed counts inherit the int32 of the inputs and wrap.
    # merge_coolers calls cooler.create internally, so it needs the same shims
    # as every other writer (see write_cooler): without them its per-contig
    # costs made merge alone take minutes at 150 k contigs. They also hand
    # cooler the inputs' known bin size, which it cannot infer once every
    # contig is a single bin.
    with cooler_writer_shims(binsize):
        cooler.merge_coolers(
            str(output),
            inputs,
            mergebuf=int(2e7),
            dtypes={"count": _merged_count_dtype(inputs)},
        )

    # Post-process: add back metacooler-specific groups
    with _open_uri(first) as fsrc, h5py.File(output, "a") as fdst:
        # Copy genomes, minus the first input's QC columns: the merged pixel
        # table is the sum of every input, so counts copied from one of them
        # would understate it while looking authoritative. Re-run
        # ``metacooler qc --store`` on the output to get honest ones.
        if "genomes" in fdst:
            del fdst["genomes"]
        fdst.copy(fsrc["genomes"], "genomes")
        for key in [k for k in fdst["genomes"] if is_qc_column("genomes", k)]:
            del fdst["genomes"][key]

        # Copy contigs (overwrites the chroms-compatible one), minus their two
        # QC columns for the same reason as the genomes' -- only those: any
        # other qc_* column there is a user's contig metadata.
        if "contigs" in fdst:
            del fdst["contigs"]
        fdst.copy(fsrc["contigs"], "contigs")
        for key in [k for k in fdst["contigs"] if is_qc_column("contigs", k)]:
            del fdst["contigs"][key]

        # Soft link chroms -> contigs
        if "chroms" in fdst:
            del fdst["chroms"]
        fdst["chroms"] = h5py.SoftLink("contigs")

        # Copy genome_offset
        if "genome_offset" in fdst["indexes"]:
            del fdst["indexes"]["genome_offset"]
        fdst["indexes"].copy(fsrc["indexes"]["genome_offset"], "genome_offset")

        # Strip cooler's enum dtype from bins/chrom — cooler.merge_coolers()
        # always writes bins/chrom as an HDF5 enum type, which is inconsistent
        # with the metacooler schema that stores it as plain int32.
        chrom_data = fdst["bins"]["chrom"][:]
        del fdst["bins"]["chrom"]
        write_column(fdst["bins"], "chrom", chrom_data, dtype=np.int32)

        # Set attrs
        fdst.attrs["format"] = COOLER_FORMAT
        fdst.attrs["format-url"] = COOLER_FORMAT_URL
        fdst.attrs["metacooler-format"] = METACOOLER_FORMAT
        fdst.attrs["metacooler-format-url"] = METACOOLER_FORMAT_URL
        fdst.attrs["metacooler-format-version"] = METACOOLER_FORMAT_VERSION
        fdst.attrs["generated-by"] = f"metacooler-{__version__}"
        fdst.attrs["ngenomes"] = len(fsrc["genomes"]["name"])
        fdst.attrs["contig-ordering"] = ordering
        # The inputs were checked to share one bin grid, so its description
        # carries over verbatim rather than being re-inferred from the table.
        for key in ("bin-size", "bin-type"):
            if key in fsrc.attrs:
                fdst.attrs[key] = fsrc.attrs[key]

        # Same reasoning as the QC columns above: whatever block an input
        # carried describes that input alone.
        raw = fdst.attrs.get("metadata")
        if raw is not None:
            if isinstance(raw, bytes):
                raw = raw.decode("utf-8")
            meta = json.loads(raw) if isinstance(raw, str) else dict(raw)
            if QC_METADATA_KEY in meta:
                del meta[QC_METADATA_KEY]
                fdst.attrs["metadata"] = json.dumps(meta)

    validate(output)

Metacooler collections

MMetaCooler I/O operations — create, add, remove, copy, move, list.

init_mmetacool_attrs

init_mmetacool_attrs(f: File) -> None

Initialise an open HDF5 file as an empty .mmetacool collection.

Sets format to HDF5::MMetaCooler and format-version to the current :data:~metacooler.schema.MMETACOOLER_FORMAT_VERSION, and creates the top-level /entries group. Both are required by :func:metacooler.schema.is_mmetacooler, so a file initialised through this function is a valid (empty) mmetacool.

Parameters:

Name Type Description Default
f File

Open h5py.File in write or append mode. Attributes are written directly to the file root (f.attrs).

required
Source code in src/metacooler/io/mmetacool.py
def init_mmetacool_attrs(f: h5py.File) -> None:
    """Initialise an open HDF5 file as an empty ``.mmetacool`` collection.

    Sets ``format`` to ``HDF5::MMetaCooler`` and ``format-version`` to the
    current :data:`~metacooler.schema.MMETACOOLER_FORMAT_VERSION`, and
    creates the top-level ``/entries`` group.  Both are required by
    :func:`metacooler.schema.is_mmetacooler`, so a file initialised through
    this function is a valid (empty) mmetacool.

    Args:
        f: Open ``h5py.File`` in write or append mode.  Attributes are
            written directly to the file root (``f.attrs``).
    """
    f.attrs["format"] = MMETACOOLER_FORMAT
    f.attrs["format-version"] = MMETACOOLER_FORMAT_VERSION
    f.require_group("entries")

create_mmetacool

create_mmetacool(
    output: Path, entries: dict[str, Path], force: bool = False
) -> None

Create a new .mmetacool file from a dict of {name: metacool_path}.

Each entry is a full metacooler stored as an HDF5 group under /entries/<name>.

Parameters:

Name Type Description Default
output Path

Destination path for the .mmetacool file.

required
entries dict[str, Path]

Mapping of entry names to .metacool file paths.

required
force bool

If True, overwrite an existing output file.

False

Raises:

Type Description
FileExistsError

If output already exists and force is False.

FileNotFoundError

If any source metacool file does not exist.

Source code in src/metacooler/io/mmetacool.py
def create_mmetacool(
    output: Path,
    entries: dict[str, Path],
    force: bool = False,
) -> None:
    """Create a new ``.mmetacool`` file from a dict of ``{name: metacool_path}``.

    Each entry is a full metacooler stored as an HDF5 group under
    ``/entries/<name>``.

    Args:
        output: Destination path for the ``.mmetacool`` file.
        entries: Mapping of entry names to ``.metacool`` file paths.
        force: If ``True``, overwrite an existing output file.

    Raises:
        FileExistsError: If *output* already exists and *force* is ``False``.
        FileNotFoundError: If any source metacool file does not exist.
    """
    if output.exists() and not force:
        raise FileExistsError(
            f"Output file already exists: {output}. Pass force=True to overwrite."
        )
    for name, src in entries.items():
        _check_entry_name(name)
        if not Path(src).exists():
            raise FileNotFoundError(f"Source file not found for entry {name!r}: {src}")
        _require_metacool_source(str(src))

    with h5py.File(output, "w") as dst:
        init_mmetacool_attrs(dst)
        for name, src in entries.items():
            logger.info("Adding entry %r from %s", name, src)
            with h5py.File(src, "r") as src_f:
                _copy_map(src_f["/"], dst["entries"], name)

    logger.info("Created mmetacool with %d entries: %s", len(entries), output)

add_entry

add_entry(
    mmetacool: Path, name: str, metacool: Path, overwrite: bool = False
) -> None

Add a metacooler entry to an existing .mmetacool file.

Parameters:

Name Type Description Default
mmetacool Path

Path to the .mmetacool file.

required
name str

Name for the new entry (stored under /entries/<name>).

required
metacool Path

Path to the source .metacool file.

required
overwrite bool

If True, replace an existing entry with the same name.

False

Raises:

Type Description
FileNotFoundError

If metacool or mmetacool does not exist.

ValueError

If mmetacool is not a valid mmetacool file.

KeyError

If name already exists and overwrite is False.

Source code in src/metacooler/io/mmetacool.py
def add_entry(
    mmetacool: Path,
    name: str,
    metacool: Path,
    overwrite: bool = False,
) -> None:
    """Add a metacooler entry to an existing ``.mmetacool`` file.

    Args:
        mmetacool: Path to the ``.mmetacool`` file.
        name: Name for the new entry (stored under ``/entries/<name>``).
        metacool: Path to the source ``.metacool`` file.
        overwrite: If True, replace an existing entry with the same name.

    Raises:
        FileNotFoundError: If *metacool* or *mmetacool* does not exist.
        ValueError: If *mmetacool* is not a valid mmetacool file.
        KeyError: If *name* already exists and *overwrite* is False.
    """
    _check_entry_name(name)
    if not Path(metacool).exists():
        raise FileNotFoundError(f"Source file not found: {metacool}")
    _require_metacool_source(str(metacool))
    mmetacool = _require_mmetacool(mmetacool)

    with h5py.File(mmetacool, "a") as dst:
        grp = dst.require_group("entries")
        if name in grp:
            if not overwrite:
                raise KeyError(
                    f"Entry {name!r} already exists. Use overwrite=True to replace."
                )
            del grp[name]
        with h5py.File(metacool, "r") as src_f:
            _copy_map(src_f["/"], grp, name)
    logger.info("Added entry %r to %s", name, mmetacool)

remove_entry

remove_entry(mmetacool: Path, name: str) -> None

Remove an entry from an .mmetacool file.

Parameters:

Name Type Description Default
mmetacool Path

Path to the .mmetacool file.

required
name str

Name of the entry to remove.

required

Raises:

Type Description
FileNotFoundError

If mmetacool does not exist.

ValueError

If mmetacool is not a valid mmetacool file.

KeyError

If name does not exist.

Source code in src/metacooler/io/mmetacool.py
def remove_entry(mmetacool: Path, name: str) -> None:
    """Remove an entry from an ``.mmetacool`` file.

    Args:
        mmetacool: Path to the ``.mmetacool`` file.
        name: Name of the entry to remove.

    Raises:
        FileNotFoundError: If *mmetacool* does not exist.
        ValueError: If *mmetacool* is not a valid mmetacool file.
        KeyError: If *name* does not exist.
    """
    mmetacool = _require_mmetacool(mmetacool)
    with h5py.File(mmetacool, "a") as f:
        grp = f.require_group("entries")
        if name not in grp:
            raise KeyError(f"Entry {name!r} not found in {mmetacool}")
        del grp[name]
    logger.info("Removed entry %r from %s", name, mmetacool)

copy_entry

copy_entry(
    mmetacool: Path, src: str, dst: str, overwrite: bool = False
) -> None

Copy an entry within the same .mmetacool file.

Parameters:

Name Type Description Default
mmetacool Path

Path to the .mmetacool file.

required
src str

Source entry name.

required
dst str

Destination entry name.

required
overwrite bool

If True, replace an existing destination entry.

False

Raises:

Type Description
FileNotFoundError

If mmetacool does not exist.

KeyError

If src does not exist or dst exists and overwrite is False.

ValueError

If src and dst are the same, or mmetacool is not a valid mmetacool file.

Source code in src/metacooler/io/mmetacool.py
def copy_entry(
    mmetacool: Path,
    src: str,
    dst: str,
    overwrite: bool = False,
) -> None:
    """Copy an entry within the same ``.mmetacool`` file.

    Args:
        mmetacool: Path to the ``.mmetacool`` file.
        src: Source entry name.
        dst: Destination entry name.
        overwrite: If True, replace an existing destination entry.

    Raises:
        FileNotFoundError: If *mmetacool* does not exist.
        KeyError: If *src* does not exist or *dst* exists and *overwrite* is False.
        ValueError: If *src* and *dst* are the same, or *mmetacool* is not a
            valid mmetacool file.
    """
    _check_entry_name(dst)
    if src == dst:
        raise ValueError("Source and destination cannot be the same.")
    mmetacool = _require_mmetacool(mmetacool)

    with h5py.File(mmetacool, "a") as f:
        grp = f.require_group("entries")
        if src not in grp:
            raise KeyError(f"Source entry {src!r} not found.")
        if dst in grp:
            if not overwrite:
                raise KeyError(
                    f"Destination entry {dst!r} already exists. Use overwrite=True."
                )
            del grp[dst]
        _copy_map(grp[src], grp, dst)
    logger.info("Copied entry %r → %r in %s", src, dst, mmetacool)

move_entry

move_entry(
    mmetacool: Path, src: str, dst: str, overwrite: bool = False
) -> None

Move (rename) an entry within the same .mmetacool file.

Implemented as an HDF5 link rename (:meth:h5py.Group.move), which is O(1) and rewrites no data. Copying then deleting instead would need twice the peak disk space and would leave the freed space stranded — HDF5 never returns deleted space to the filesystem, so a rename would permanently grow the file by the size of the entry.

Parameters:

Name Type Description Default
mmetacool Path

Path to the .mmetacool file.

required
src str

Source entry name.

required
dst str

Destination entry name.

required
overwrite bool

If True, replace an existing destination entry.

False

Raises:

Type Description
FileNotFoundError

If mmetacool does not exist.

KeyError

If src does not exist or dst exists and overwrite is False.

ValueError

If src and dst are the same, or mmetacool is not a valid mmetacool file.

Source code in src/metacooler/io/mmetacool.py
def move_entry(
    mmetacool: Path,
    src: str,
    dst: str,
    overwrite: bool = False,
) -> None:
    """Move (rename) an entry within the same ``.mmetacool`` file.

    Implemented as an HDF5 link rename (:meth:`h5py.Group.move`), which is
    O(1) and rewrites no data.  Copying then deleting instead would need
    twice the peak disk space and would leave the freed space stranded —
    HDF5 never returns deleted space to the filesystem, so a rename would
    permanently grow the file by the size of the entry.

    Args:
        mmetacool: Path to the ``.mmetacool`` file.
        src: Source entry name.
        dst: Destination entry name.
        overwrite: If True, replace an existing destination entry.

    Raises:
        FileNotFoundError: If *mmetacool* does not exist.
        KeyError: If *src* does not exist or *dst* exists and *overwrite* is False.
        ValueError: If *src* and *dst* are the same, or *mmetacool* is not a
            valid mmetacool file.
    """
    _check_entry_name(dst)
    if src == dst:
        raise ValueError("Source and destination cannot be the same.")
    mmetacool = _require_mmetacool(mmetacool)

    with h5py.File(mmetacool, "a") as f:
        grp = f.require_group("entries")
        if src not in grp:
            raise KeyError(f"Source entry {src!r} not found.")
        if dst in grp:
            if not overwrite:
                raise KeyError(
                    f"Destination entry {dst!r} already exists. Use overwrite=True."
                )
            del grp[dst]
        grp.move(src, dst)
    logger.info("Moved entry %r → %r in %s", src, dst, mmetacool)

list_entries

list_entries(mmetacool: Path) -> list[str]

List all entry names in an .mmetacool file.

Both this and :func:metacooler.schema.list_metacoolers are thin conventions over the single scan in :func:metacooler.schema.list_entry_names: this one yields bare entry names ("sample1"), that one yields URI-ready group paths ("/entries/sample1").

Parameters:

Name Type Description Default
mmetacool Path

Path to the .mmetacool file.

required

Returns:

Type Description
list[str]

A list of entry names (keys under /entries), or an empty list if

list[str]

the file has no entries group.

Source code in src/metacooler/io/mmetacool.py
def list_entries(mmetacool: Path) -> list[str]:
    """List all entry names in an ``.mmetacool`` file.

    Both this and :func:`metacooler.schema.list_metacoolers` are thin
    conventions over the single scan in
    :func:`metacooler.schema.list_entry_names`: this one yields bare entry
    names (``"sample1"``), that one yields URI-ready group paths
    (``"/entries/sample1"``).

    Args:
        mmetacool: Path to the ``.mmetacool`` file.

    Returns:
        A list of entry names (keys under ``/entries``), or an empty list if
        the file has no ``entries`` group.
    """
    return list_entry_names(mmetacool)

copy_entry_cross_file

copy_entry_cross_file(
    src_file: Path,
    src_name: str,
    dst_file: Path,
    dst_name: str,
    overwrite: bool = False,
) -> None

Copy an entry from one .mmetacool file to another.

Parameters:

Name Type Description Default
src_file Path

Source .mmetacool file path.

required
src_name str

Entry name in the source file.

required
dst_file Path

Destination .mmetacool file path.

required
dst_name str

Entry name in the destination file.

required
overwrite bool

If True, replace an existing destination entry.

False

Raises:

Type Description
FileNotFoundError

If src_file or dst_file does not exist.

KeyError

If source entry does not exist or destination exists and overwrite is False.

ValueError

If source and destination resolve to the same file, or either is not a valid mmetacool file.

Source code in src/metacooler/io/mmetacool.py
def copy_entry_cross_file(
    src_file: Path,
    src_name: str,
    dst_file: Path,
    dst_name: str,
    overwrite: bool = False,
) -> None:
    """Copy an entry from one ``.mmetacool`` file to another.

    Args:
        src_file: Source ``.mmetacool`` file path.
        src_name: Entry name in the source file.
        dst_file: Destination ``.mmetacool`` file path.
        dst_name: Entry name in the destination file.
        overwrite: If True, replace an existing destination entry.

    Raises:
        FileNotFoundError: If *src_file* or *dst_file* does not exist.
        KeyError: If source entry does not exist or destination exists
            and *overwrite* is False.
        ValueError: If source and destination resolve to the same file, or
            either is not a valid mmetacool file.
    """
    _check_entry_name(dst_name)
    if Path(src_file).resolve() == Path(dst_file).resolve():
        raise ValueError(
            f"Source and destination are the same file: {src_file}. "
            "Use rename_entry() to rename entries within a file."
        )
    src_file = _require_mmetacool(src_file)
    dst_file = _require_mmetacool(dst_file)
    with h5py.File(src_file, "r") as sf:
        if "entries" not in sf or src_name not in sf["entries"]:
            raise KeyError(f"Source entry {src_name!r} not found in {src_file}")

        with h5py.File(dst_file, "a") as df:
            grp = df.require_group("entries")
            if dst_name in grp:
                if not overwrite:
                    raise KeyError(
                        f"Destination entry {dst_name!r} already exists in {dst_file}."
                    )
                del grp[dst_name]
            _copy_map(sf[f"entries/{src_name}"], grp, dst_name)

    logger.info("Copied %s::%s → %s::%s", src_file, src_name, dst_file, dst_name)

Extracting subsets

Extract a genome-level cool file from a metacool file.

extract

extract(
    uri: str | Path,
    genomes: list[str] | None = None,
    contigs: list[str] | None = None,
    filters: list[str] | None = None,
    output: str | Path = "extracted.cool",
    split: bool = False,
    force: bool = False,
) -> None

Extract selected genomes and/or contigs from a metacool file.

In single-file mode, genomes and contigs can be combined: the union of all contigs from the specified genomes plus the directly specified contigs is extracted. In split mode only genomes is allowed (one file per genome); combining it with contigs is an error.

filters composes with both modes: it narrows whatever the names selected (or the whole file, when no name is given), so --split --filter 'genomes.length > 1Mb' writes one cool per genome above 1 Mb.

Parameters:

Name Type Description Default
uri str | Path

Path to the source .metacool file.

required
genomes list[str] | None

List of genome names to extract.

None
contigs list[str] | None

List of contig names to extract (single-file mode only).

None
filters list[str] | None

--filter expressions (see :mod:metacooler.filters).

None
output str | Path

Destination path. If split is True, treated as a filename prefix (one <prefix>_<genome>.cool per genome).

'extracted.cool'
split bool

If True, write one .cool per genome instead of a single merged file. Rejects contigs.

False
force bool

Overwrite existing per-genome output files (split mode).

False

Raises:

Type Description
KeyError

If any genome or contig is not found.

ValueError

If no selector at all is specified, or if split is combined with contigs.

Source code in src/metacooler/io/extract.py
def extract(
    uri: str | Path,
    genomes: list[str] | None = None,
    contigs: list[str] | None = None,
    filters: list[str] | None = None,
    output: str | Path = "extracted.cool",
    split: bool = False,
    force: bool = False,
) -> None:
    """Extract selected genomes and/or contigs from a metacool file.

    In single-file mode, ``genomes`` and ``contigs`` can be combined: the
    union of all contigs from the specified genomes plus the directly
    specified contigs is extracted.  In ``split`` mode only ``genomes`` is
    allowed (one file per genome); combining it with ``contigs`` is an error.

    ``filters`` composes with both modes: it narrows whatever the names
    selected (or the whole file, when no name is given), so ``--split
    --filter 'genomes.length > 1Mb'`` writes one cool per genome above 1 Mb.

    Args:
        uri: Path to the source ``.metacool`` file.
        genomes: List of genome names to extract.
        contigs: List of contig names to extract (single-file mode only).
        filters: ``--filter`` expressions (see :mod:`metacooler.filters`).
        output: Destination path. If *split* is True, treated as a
            filename prefix (one ``<prefix>_<genome>.cool`` per genome).
        split: If True, write one ``.cool`` per genome instead of a
            single merged file. Rejects ``contigs``.
        force: Overwrite existing per-genome output files (split mode).

    Raises:
        KeyError: If any genome or contig is not found.
        ValueError: If no selector at all is specified, or if ``split`` is
            combined with ``contigs``.
    """
    from metacooler.api import MetaCooler

    if not genomes and not contigs and not filters:
        raise ValueError(
            "At least one of --genomes, --contigs or --filter must be specified."
        )
    if split and contigs:
        raise ValueError(
            "--contigs cannot be combined with --split. Split writes one file per "
            "genome; pass --genomes only, or drop --split to extract the contigs "
            "into a single file."
        )

    mc = MetaCooler(uri)
    selected_contig_ids = resolve_contig_selection(
        mc, genomes=genomes, contigs=contigs, filters=filters
    )
    assert selected_contig_ids is not None  # a selector was required above

    if split:
        # One cool per genome, over the selected contigs only: with --genomes
        # that is exactly the named genomes, and with --filter it is whichever
        # genomes still have a contig left.
        all_contigs = mc.contigs
        genome_names = mc.genomes["name"]
        output_path = Path(output)
        # Strip only a trailing ".cool": Path.stem cut at the *last* dot, so
        # "-o sample.rep1" wrote "sample_<genome>.cool", and a second run with
        # "-o sample.rep2" collided with (or under --force, overwrote) it.
        name = output_path.name
        prefix = name[: -len(".cool")] if name.endswith(".cool") else name
        by_genome: dict[int, list[int]] = {}
        for cid in sorted(selected_contig_ids):
            by_genome.setdefault(int(all_contigs.at[cid, "genome_id"]), []).append(cid)
        for gid, cids in sorted(by_genome.items()):
            g = str(genome_names[gid])
            out_file = output_path.parent / f"{prefix}_{g}.cool"
            if out_file.exists() and not force:
                raise FileExistsError(
                    f"Output file already exists: {out_file}. Use --force to overwrite."
                )
            _extract_contig_ids(mc, cids, out_file, assembly=g)
    else:
        # Single output file with all selected contigs
        _extract_contig_ids(
            mc, sorted(selected_contig_ids), Path(output), assembly="multi"
        )

Dumping tables

Dump helpers for metacooler tables.

dump_table

dump_table(
    uri: str | Path,
    table: Literal[
        "bins", "chroms", "contigs", "genomes", "pixels", "indexes", "all"
    ],
    genomes: list[str] | None = None,
    contigs: list[str] | None = None,
    filters: list[str] | None = None,
    range1: str | None = None,
    range2: str | None = None,
    header: bool = False,
    out: str | Path | None = None,
    prefix: str | None = None,
    pixels_format: Literal["long", "wide"] = "long",
    weight: str = "count",
    join: bool = False,
    force: bool = False,
) -> None

Dump a metacooler table to TSV.

Parameters:

Name Type Description Default
uri str | Path

Path to the metacool file.

required
table Literal['bins', 'chroms', 'contigs', 'genomes', 'pixels', 'indexes', 'all']

Which table to dump. Use "all" to dump all tables.

required
genomes list[str] | None

Multiple genome filter.

None
contigs list[str] | None

Multiple contig filter.

None
filters list[str] | None

--filter expressions (see :mod:metacooler.filters), e.g. ["contigs.length >= 10kb"]. ANDed with each other and applied within the genomes/contigs selection rather than beside it.

None
range1 str | None

Row-dimension range selector (genome, contig, or contig:start-end). Applied only when table="pixels".

None
range2 str | None

Column-dimension range selector. Defaults to range1 when omitted. Applied only when table="pixels".

None
header bool

Include header row.

False
out str | Path | None

Output file path (default: stdout).

None
prefix str | None

Required when table="all". File prefix for output files.

None
pixels_format Literal['long', 'wide']

"long" (default) or "wide" (2D matrix).

'long'
weight str

Column name for wide format values (default: "count").

'count'
join bool

Annotate pixel output with contig names and coordinates. For pixels_format="long" this replaces bin1_id/bin2_id with contig1/start1/end1/contig2/start2/end2 (cooler --join semantics); for "wide" it labels the matrix rows/columns with contig:start-end. Applied only when table="pixels".

False
force bool

Overwrite existing output file(s). With table="all" every <prefix>.<table>.tsv is guarded.

False

Raises:

Type Description
FileExistsError

If an output file exists and force is False.

ValueError

If table is unknown, if table="all" is used without prefix, or if the wide-format selection is too large to materialise densely.

Source code in src/metacooler/io/dump.py
def dump_table(
    uri: str | Path,
    table: Literal["bins", "chroms", "contigs", "genomes", "pixels", "indexes", "all"],
    genomes: list[str] | None = None,
    contigs: list[str] | None = None,
    filters: list[str] | None = None,
    range1: str | None = None,
    range2: str | None = None,
    header: bool = False,
    out: str | Path | None = None,
    prefix: str | None = None,
    pixels_format: Literal["long", "wide"] = "long",
    weight: str = "count",
    join: bool = False,
    force: bool = False,
) -> None:
    """Dump a metacooler table to TSV.

    Args:
        uri: Path to the metacool file.
        table: Which table to dump. Use ``"all"`` to dump all tables.
        genomes: Multiple genome filter.
        contigs: Multiple contig filter.
        filters: ``--filter`` expressions (see :mod:`metacooler.filters`), e.g.
            ``["contigs.length >= 10kb"]``.  ANDed with each other and applied
            *within* the *genomes*/*contigs* selection rather than beside it.
        range1: Row-dimension range selector (genome, contig, or
            ``contig:start-end``).  Applied only when ``table="pixels"``.
        range2: Column-dimension range selector.  Defaults to *range1* when
            omitted.  Applied only when ``table="pixels"``.
        header: Include header row.
        out: Output file path (default: stdout).
        prefix: Required when ``table="all"``. File prefix for output files.
        pixels_format: ``"long"`` (default) or ``"wide"`` (2D matrix).
        weight: Column name for wide format values (default: ``"count"``).
        join: Annotate pixel output with contig names and coordinates.  For
            ``pixels_format="long"`` this replaces ``bin1_id``/``bin2_id`` with
            ``contig1/start1/end1``/``contig2/start2/end2`` (cooler ``--join``
            semantics); for ``"wide"`` it labels the matrix rows/columns with
            ``contig:start-end``.  Applied only when ``table="pixels"``.
        force: Overwrite existing output file(s).  With ``table="all"`` every
            ``<prefix>.<table>.tsv`` is guarded.

    Raises:
        FileExistsError: If an output file exists and *force* is False.
        ValueError: If *table* is unknown, if ``table="all"`` is used without
            *prefix*, or if the wide-format selection is too large to
            materialise densely.
    """
    # Merge single/multi genome/contig filters
    genome_list = list(genomes) if genomes else []
    contig_list = list(contigs) if contigs else []
    filter_list = list(filters) if filters else []
    mc = MetaCooler(uri)

    if table == "all":
        if not prefix:
            raise ValueError("--prefix is required when --table all is specified.")
        # ``chroms`` is intentionally omitted: it is the cooler soft-link alias
        # of ``contigs[["name", "length"]]`` and would duplicate the contigs file.
        tables = ("genomes", "contigs", "bins", "pixels", "indexes")
        # Guard every destination up front so a clash on the last table does
        # not leave the first four already overwritten.
        for tbl in tables:
            check_output_path(Path(f"{prefix}.{tbl}.tsv"), force, inputs=[uri])
        for tbl in tables:
            out_path = Path(f"{prefix}.{tbl}.tsv")
            dump_table(
                uri=uri,
                table=tbl,  # type: ignore[arg-type]
                genomes=genome_list or None,
                contigs=contig_list or None,
                filters=filter_list or None,
                range1=range1,
                range2=range2,
                header=header,
                out=out_path,
                pixels_format=pixels_format,
                weight=weight,
                join=join,
                force=True,
            )
        return

    if out is not None:
        check_output_path(out, force, inputs=[uri])

    # Resolve contig filter
    selected_contig_ids = resolve_contig_selection(
        mc,
        genomes=genome_list or None,
        contigs=contig_list or None,
        filters=filter_list or None,
    )

    write_index = False
    if table == "bins":
        df = mc.bins()
        if selected_contig_ids is not None:
            bin_chrom_codes = chrom_codes(df)
            mask = np.isin(bin_chrom_codes, list(selected_contig_ids))
            df = df[mask]
        df = df.copy()
        df["chrom"] = _decode_chrom(df["chrom"])
    elif table == "chroms":
        df = mc.contigs[["name", "length"]]
        if selected_contig_ids is not None:
            df = df.loc[df.index.isin(selected_contig_ids)]
    elif table == "contigs":
        df = mc.contigs.copy()
        if selected_contig_ids is not None:
            df = df.loc[df.index.isin(selected_contig_ids)]
        # Surface the genome/MAG name alongside the internal integer genome_id.
        df["genome"] = df["genome_id"].map(mc.genomes["name"])
        core = ["name", "length", "genome", "genome_id"]
        extras = [c for c in df.columns if c not in core]
        df = df[core + extras]
    elif table == "genomes":
        df = mc.genomes
        if selected_contig_ids is not None:
            sel_genome_ids = set(mc.contigs.loc[list(selected_contig_ids), "genome_id"])
            df = df.loc[df.index.isin(sel_genome_ids)]
        core = ["name", "length", "n_contigs"]
        extras = [c for c in df.columns if c not in core]
        df = df[core + extras]
    elif table == "pixels":
        # Pre-compute filter parameters that are independent of chunk
        valid_bin_ids: np.ndarray | None = None
        if selected_contig_ids is not None:
            # Through chrom_offset, not the bin table.
            valid_bin_ids = _ranges_to_ids(
                mc._contig_bin_ranges(np.array(sorted(selected_contig_ids)))
            )

        range_bounds: tuple[int, int, int, int] | None = None
        if range1 is not None:
            from metacooler.ops.range import parse_metacooler_range

            i0, i1 = parse_metacooler_range(mc, range1)
            j0, j1 = parse_metacooler_range(mc, range2) if range2 else (i0, i1)
            range_bounds = (i0, i1, j0, j1)

        if pixels_format == "wide":
            pixel_columns = list(mc._empty_pixels().columns)
            if weight not in pixel_columns:
                raise ValueError(
                    f"Weight column {weight!r} not found. Available: {pixel_columns}"
                )

            # Resolve the *selected* bin set before touching the pixel table:
            # the dense matrix is allocated over that set only, and the size
            # guard must fire before anything is materialised.
            n_bins = mc.shape[0]
            selected_bins = np.arange(n_bins, dtype=np.int64)
            if valid_bin_ids is not None:
                selected_bins = np.asarray(valid_bin_ids, dtype=np.int64)
            if range_bounds is not None:
                i0, i1, j0, j1 = range_bounds
                # A range selection keeps pixels in the row block, the column
                # block, or their transpose, so the axis is the union of both.
                in_range = np.union1d(
                    np.arange(i0, i1, dtype=np.int64),
                    np.arange(j0, j1, dtype=np.int64),
                )
                selected_bins = np.intersect1d(
                    selected_bins, in_range, assume_unique=True
                )
            n_sel = int(selected_bins.size)
            if n_sel > DENSE_MATRIX_BIN_LIMIT:
                raise ValueError(
                    f"Wide format refuses to allocate a {n_sel}\u00d7{n_sel} matrix "
                    f"({n_sel**2 * 8 / 1e9:.1f} GB). Use --pixels-format long."
                )

            # Global bin id -> position along the selected axis (-1 = dropped).
            local_of_bin = np.full(n_bins, -1, dtype=np.int64)
            local_of_bin[selected_bins] = np.arange(n_sel, dtype=np.int64)

            mat = np.zeros((n_sel, n_sel), dtype=np.float64)
            for chunk in _selected_pixels(mc, valid_bin_ids, range_bounds):
                b1 = local_of_bin[chunk["bin1_id"].to_numpy(dtype=np.int64)]
                b2 = local_of_bin[chunk["bin2_id"].to_numpy(dtype=np.int64)]
                keep = (b1 >= 0) & (b2 >= 0)
                if not keep.any():
                    continue
                b1, b2 = b1[keep], b2[keep]
                vals = chunk[weight].to_numpy()[keep].astype(np.float64)
                mat[b1, b2] = vals
                mat[b2, b1] = vals
            if join:
                bdf = mc.bins()[["chrom", "start", "end"]].iloc[selected_bins]
                chrom = _decode_chrom(bdf["chrom"]).astype(str).to_numpy()
                labels = [
                    f"{c}:{s}-{e}"
                    for c, s, e in zip(
                        chrom,
                        bdf["start"].to_numpy(),
                        bdf["end"].to_numpy(),
                        strict=False,
                    )
                ]
                df = pd.DataFrame(mat, index=labels, columns=labels)
                write_index = True
            else:
                df = pd.DataFrame(mat)
        else:
            # Long format: stream pixel chunks to output without full materialisation
            ann: pd.DataFrame | None = None
            joined_empty_cols: list[str] | None = None
            if join:
                import cooler

                ann = mc.bins()[["chrom", "start", "end"]].copy()
                ann["chrom"] = _decode_chrom(ann["chrom"])
                pixel_cols = [
                    c
                    for c in mc._empty_pixels().columns
                    if c not in ("bin1_id", "bin2_id")
                ]
                joined_empty_cols = [
                    "contig1",
                    "start1",
                    "end1",
                    "contig2",
                    "start2",
                    "end2",
                    *pixel_cols,
                ]
            out_stream = open(out, "w") if out else sys.stdout  # noqa: SIM115
            try:
                first_chunk = True
                for chunk in _selected_pixels(mc, valid_bin_ids, range_bounds):
                    if join:
                        assert ann is not None
                        chunk = cooler.api.annotate(chunk, ann, replace=True).rename(
                            columns={"chrom1": "contig1", "chrom2": "contig2"}
                        )
                    chunk.to_csv(
                        out_stream,
                        sep="\t",
                        index=False,
                        header=header and first_chunk,
                    )
                    first_chunk = False
                if first_chunk and header:
                    # Nothing matched: still honour --header.
                    empty = (
                        pd.DataFrame(columns=joined_empty_cols)
                        if join
                        else mc._empty_pixels()
                    )
                    empty.to_csv(out_stream, sep="\t", index=False, header=True)
            finally:
                if out:
                    out_stream.close()
            return
    elif table == "indexes":
        rows = []
        with mc.open() as f:
            idx = f["indexes"]
            for key in idx:
                arr = idx[key][:]
                for i, val in enumerate(arr):
                    rows.append({"index": key, "position": i, "value": val})
        df = pd.DataFrame(rows)
    else:
        raise ValueError(f"Unknown table: {table!r}")

    if out:
        with open(out, "w") as out_stream:
            df.to_csv(
                out_stream, sep="\t", index=write_index, header=header or write_index
            )
    else:
        df.to_csv(sys.stdout, sep="\t", index=write_index, header=header or write_index)

Flattening to a cooler

Flatten a metacool file into a standard cool file.

flatten

flatten(uri: str | Path, output: str | Path) -> None

Flatten a metacool file into a standard cool file.

Removes the genome hierarchy: all contigs become "chromosomes" in the output cool file. Bins and pixels are preserved as-is (bin IDs do not change since contigs are already the chrom column).

Parameters:

Name Type Description Default
uri str | Path

Path to the source .metacool file.

required
output str | Path

Destination path for the output .cool file.

required
Source code in src/metacooler/io/flatten.py
def flatten(
    uri: str | Path,
    output: str | Path,
) -> None:
    """Flatten a metacool file into a standard cool file.

    Removes the genome hierarchy: all contigs become "chromosomes" in
    the output cool file.  Bins and pixels are preserved as-is (bin IDs
    do not change since contigs are already the ``chrom`` column).

    Args:
        uri: Path to the source ``.metacool`` file.
        output: Destination path for the output ``.cool`` file.
    """
    from metacooler.api import MetaCooler

    mc = MetaCooler(uri)

    # Get contigs as the chromosomes for the output cool
    contigs = mc.contigs

    # Build bins DataFrame — chrom column already uses contig names
    bins = mc.bins().copy()
    # Ensure chrom is string, not categorical codes
    if hasattr(bins["chrom"], "cat"):
        bins["chrom"] = bins["chrom"].astype(str)

    bins = bins[["chrom", "start", "end"]].copy()
    bins["start"] = bins["start"].astype(np.int32)
    bins["end"] = bins["end"].astype(np.int32)

    # Stream pixels in chunks to avoid loading the full table into memory.
    chunksize = DEFAULT_CHUNKSIZE
    n_pix = mc._clr.info["nnz"]

    # Preserve the width of the source ``count`` column: a binnified source
    # writes int64 and would silently wrap if narrowed back to int32 here.
    count_dtype = resolve_count_dtype(
        mc._clr.pixels()[:0]["count"].dtype, context="flatten"
    )

    def _pixel_stream():
        for _i in range(0, n_pix, chunksize):
            chunk = mc._clr.pixels()[_i : _i + chunksize].copy()
            chunk["bin1_id"] = chunk["bin1_id"].astype(np.int64)
            chunk["bin2_id"] = chunk["bin2_id"].astype(np.int64)
            chunk["count"] = chunk["count"].astype(count_dtype)
            enforce_upper_triangle(chunk)
            yield chunk[["bin1_id", "bin2_id", "count"]]

    logger.info(
        "Flattening %s → %s: %d contigs, %d bins",
        uri,
        output,
        len(contigs),
        len(bins),
    )

    write_cooler(
        output,
        bins,
        _pixel_stream(),
        ordered=True,
        symmetric_upper=True,
        dtypes={"count": count_dtype},
        metadata={"source-file": str(uri), "flattened": True},
    )
    logger.info("Written flattened cool to %s", output)

Exporting network

Network edge list export from metacool files.

network

network(
    uri: str | Path,
    level: Literal["bin", "contig", "genome"] = "contig",
    weight: str = "count",
    out: str | Path | None = None,
    header: bool = False,
    chunksize: int = DEFAULT_CHUNKSIZE,
    force: bool = False,
) -> None

Export a network edge list from a metacool file.

Parameters:

Name Type Description Default
uri str | Path

Path to the source .metacool file.

required
level Literal['bin', 'contig', 'genome']

Aggregation level — bin, contig, or genome.

'contig'
weight str

Column name in the pixels table to use as edge weight.

'count'
out str | Path | None

Output file path. If None, writes to stdout.

None
header bool

If True, write a node1\tnode2\tvalue header line.

False
chunksize int

Number of pixels to process per chunk.

DEFAULT_CHUNKSIZE
force bool

Overwrite out if it already exists.

False

Raises:

Type Description
ValueError

If weight column does not exist in the pixels table, or if level is not one of bin/contig/genome.

FileExistsError

If out exists and force is False.

Source code in src/metacooler/io/network.py
def network(
    uri: str | Path,
    level: Literal["bin", "contig", "genome"] = "contig",
    weight: str = "count",
    out: str | Path | None = None,
    header: bool = False,
    chunksize: int = DEFAULT_CHUNKSIZE,
    force: bool = False,
) -> None:
    """Export a network edge list from a metacool file.

    Args:
        uri: Path to the source ``.metacool`` file.
        level: Aggregation level — ``bin``, ``contig``, or ``genome``.
        weight: Column name in the pixels table to use as edge weight.
        out: Output file path. If None, writes to stdout.
        header: If ``True``, write a ``node1\\tnode2\\tvalue`` header line.
        chunksize: Number of pixels to process per chunk.
        force: Overwrite *out* if it already exists.

    Raises:
        ValueError: If *weight* column does not exist in the pixels table, or
            if *level* is not one of ``bin``/``contig``/``genome``.
        FileExistsError: If *out* exists and *force* is False.
    """
    from metacooler.api import MetaCooler

    mc = MetaCooler(uri)

    # Validate weight column
    with mc.open() as f:
        pixel_cols = list(f["pixels"].keys())
    if weight not in pixel_cols:
        raise ValueError(
            f"Weight column {weight!r} not found in pixels table. "
            f"Available columns: {pixel_cols}"
        )
    if level not in ("bin", "contig", "genome"):
        raise ValueError(f"level must be 'bin', 'contig', or 'genome', got {level!r}")

    if out is not None:
        check_output_path(out, force, inputs=[uri])

    n_pix = len(mc._clr.pixels())

    if level == "bin":
        # Raw pixel edge list — stream chunks directly to output file.
        # Wrap stdout in nullcontext so the ``with`` block does not close it
        # (closing sys.stdout would break any later output in the process).
        cm = open(out, "w") if out else contextlib.nullcontext(sys.stdout)  # noqa: SIM115
        with cm as fout:
            if header:
                fout.write("node1\tnode2\tvalue\n")
            for lo in range(0, n_pix, chunksize):
                chunk = mc._clr.pixels()[lo : lo + chunksize]
                df = pd.DataFrame(
                    {
                        "node1": chunk["bin1_id"].values,
                        "node2": chunk["bin2_id"].values,
                        "value": chunk[weight].values,
                    }
                )
                fout.write(df.to_csv(sep="\t", index=False, header=False))
        return

    # Contig and genome levels: map every bin to its node, then reduce.
    bin_to_contig = chrom_codes(mc.bins())
    if level == "contig":
        bin_to_node = bin_to_contig
        node_names = mc.contigs["name"].values
    else:
        contig_to_genome = mc.contigs["genome_id"].values
        bin_to_node = contig_to_genome[bin_to_contig]
        node_names = mc.genomes["name"].values

    agg = _aggregate_pairs(mc, bin_to_node, weight, n_pix, chunksize)
    if agg is None:
        edges = pd.DataFrame(columns=["node1", "node2", "value"])
    else:
        edges = pd.DataFrame(
            {
                "node1": node_names[agg["node1"].values],
                "node2": node_names[agg["node2"].values],
                "value": agg["value"].values,
            }
        )

    logger.info("Generated %d edges at %s level", len(edges), level)

    # Write TSV (header optional for compatibility with igraph, networkx, etc.)
    if out:
        edges.to_csv(out, sep="\t", index=False, header=header)
    else:
        edges.to_csv(sys.stdout, sep="\t", index=False, header=header)

Parsing contigs tables

Contig table parsers.

parse_contigs_table

parse_contigs_table(
    path: str | Path, fmt: str = CONTIGS_FORMAT_DEFAULT
) -> tuple[dict[str, str], pd.DataFrame]

Parse a contig-to-genome assignment table.

Parameters:

Name Type Description Default
path str | Path

Path to the contig table file.

required
fmt str

Which columns hold the contig and the genome names, in the grammar of :mod:metacooler.io.colspec: CONTIG,GENOME[,noheader], each column a 1-based position or a header name -- e.g. "Name,Final_bin", "2,5" or "1,2,noheader" -- or one of :data:~metacooler.io.colspec.CONTIGS_FORMAT_ALIASES. The default, contigs, is contig_name,genome_name: looked up by name, so a table given without its format is refused rather than misread. NA in the genome's place ("1,NA,noheader", "contig_name,NA") says the table has no genome column: every contig it lists is assigned to __unbinned__.

CONTIGS_FORMAT_DEFAULT

Returns:

Type Description
dict[str, str]

Tuple of (mapping contig_name -> genome_name, full DataFrame with

DataFrame

the two key columns relabelled contig_name / genome_id in

tuple[dict[str, str], DataFrame]

place, every other column kept -- a headerless table's as colN).

tuple[dict[str, str], DataFrame]

Contigs not present in the table are omitted from the mapping (the

tuple[dict[str, str], DataFrame]

caller decides whether to assign them to __unbinned__ or drop

tuple[dict[str, str], DataFrame]

them).

Raises:

Type Description
FileNotFoundError

If path does not exist.

ValueError

If fmt is malformed, the table is empty or misaligned, or the columns fmt names are not in it.

Source code in src/metacooler/io/contigs.py
def parse_contigs_table(
    path: str | Path,
    fmt: str = CONTIGS_FORMAT_DEFAULT,
) -> tuple[dict[str, str], pd.DataFrame]:
    """Parse a contig-to-genome assignment table.

    Args:
        path: Path to the contig table file.
        fmt: Which columns hold the contig and the genome names, in the grammar
            of :mod:`metacooler.io.colspec`: ``CONTIG,GENOME[,noheader]``, each
            column a 1-based position or a header name -- e.g.
            ``"Name,Final_bin"``, ``"2,5"`` or ``"1,2,noheader"`` -- or one of
            :data:`~metacooler.io.colspec.CONTIGS_FORMAT_ALIASES`.  The default,
            ``contigs``, is ``contig_name,genome_name``: looked up by name, so
            a table given without its format is refused rather than misread.
            ``NA`` in the genome's place (``"1,NA,noheader"``,
            ``"contig_name,NA"``) says the table has no genome column: every
            contig it lists is assigned to ``__unbinned__``.

    Returns:
        Tuple of (mapping ``contig_name -> genome_name``, full DataFrame with
        the two key columns relabelled ``contig_name`` / ``genome_id`` in
        place, every other column kept -- a headerless table's as ``colN``).
        Contigs not present in the table are omitted from the mapping (the
        caller decides whether to assign them to ``__unbinned__`` or drop
        them).

    Raises:
        FileNotFoundError: If *path* does not exist.
        ValueError: If *fmt* is malformed, the table is empty or misaligned,
            or the columns *fmt* names are not in it.
    """
    spec = parse_column_spec(
        fmt,
        keys=ASSIGNMENT_KEYS,
        aliases=CONTIGS_FORMAT_ALIASES,
        option="--contigs-format",
    )
    path = Path(path)
    if not path.exists():
        raise FileNotFoundError(f"Contig table not found: {path}")
    _check_not_empty(path, "Contig table")

    df = read_user_table(path, kind="Contig table", header=spec.header)
    contig_col, genome_col = resolve_column_spec(
        spec,
        list(df.columns),
        kind="Contig table",
        path=path,
        keys=ASSIGNMENT_KEYS,
        hint=f", or an alias: {', '.join(CONTIGS_FORMAT_ALIASES)}",
    )
    assert contig_col is not None  # parse_column_spec refuses NA in its place
    if genome_col is None:
        # NA: the table assigns no genome.  Give it a genome column that says
        # so on every row, and the rest of the pipeline reads an ordinary
        # assignment table; a column of the table's own that happens to be
        # called genome_id is moved aside by rename_key_columns, as ever.
        genome_col = df.shape[1]
        df = df.copy()
        df.insert(genome_col, "__no_genome__", "__unbinned__", allow_duplicates=True)
    # Positional, never by name: see rename_key_columns.
    df = rename_key_columns(
        df, {contig_col: "contig_name", genome_col: "genome_id"}, kind="Contig table"
    )
    _warn_duplicate_contigs(df)
    return _name_mapping(df), df

Parsing genomes tables

I/O utilities for reading genome-metadata tables.

This module provides helpers for parsing the TSV-formatted genome/MAG metadata file that describes which genomes are present in a metagenomic Hi-C dataset.

parse_genomes_table

parse_genomes_table(
    path: str | Path, fmt: str = GENOMES_FORMAT_DEFAULT
) -> pd.DataFrame

Parse a genomes metadata table.

One column holds the genome names; every other column is metadata, stored verbatim in the genomes table.

Parameters:

Name Type Description Default
path str | Path

Path to the genomes TSV file.

required
fmt str

Which column holds the genome names, in the grammar of :mod:metacooler.io.colspec: a 1-based position or a header name, plus ,noheader when the table has no header line (its metadata columns are then named col2, col3, ...). Defaults to the column named genome_name, wherever it is.

GENOMES_FORMAT_DEFAULT

Returns:

Type Description
DataFrame

DataFrame with a genome_id column (the genome names, relabelled in

DataFrame

place) and every other column of the table.

Raises:

Type Description
FileNotFoundError

If path does not exist.

ValueError

If fmt is malformed, the file is empty, its header and data rows disagree on the number of fields, or the column fmt names is not in it.

Source code in src/metacooler/io/genomes.py
def parse_genomes_table(
    path: str | Path, fmt: str = GENOMES_FORMAT_DEFAULT
) -> pd.DataFrame:
    """Parse a genomes metadata table.

    One column holds the genome names; every other column is metadata, stored
    verbatim in the ``genomes`` table.

    Args:
        path: Path to the genomes TSV file.
        fmt: Which column holds the genome names, in the grammar of
            :mod:`metacooler.io.colspec`: a 1-based position or a header name,
            plus ``,noheader`` when the table has no header line (its metadata
            columns are then named ``col2``, ``col3``, ...).  Defaults to the
            column named ``genome_name``, wherever it is.

    Returns:
        DataFrame with a ``genome_id`` column (the genome names, relabelled in
        place) and every other column of the table.

    Raises:
        FileNotFoundError: If *path* does not exist.
        ValueError: If *fmt* is malformed, the file is empty, its header and
            data rows disagree on the number of fields, or the column *fmt*
            names is not in it.
    """
    spec = parse_column_spec(fmt, keys=GENOMES_KEYS, option="--genomes-format")
    path = Path(path)
    if not path.exists():
        raise FileNotFoundError(f"Genomes table not found: {path}")
    if path.stat().st_size == 0:
        raise ValueError(f"Genomes table is empty: {path}")
    df = read_user_table(path, kind="Genomes table", header=spec.header)
    (key,) = resolve_column_spec(
        spec, list(df.columns), kind="Genomes table", path=path, keys=GENOMES_KEYS
    )
    assert key is not None  # parse_column_spec refuses NA for a genomes table
    # Positional, never by name: see rename_key_columns.
    df = rename_key_columns(df, {key: "genome_id"}, kind="Genomes table")
    dup = df["genome_id"].duplicated()
    if dup.any():
        names = df.loc[dup, "genome_id"].unique().tolist()
        logger.warning(
            "Genomes table contains %d duplicate genome name(s): %s (last row wins).",
            len(names),
            names[:10],
        )
    return df