Documentation Overview

There are two basic ways to access the DREAMS data, with a third under development: (1) directly download the data via Globus, (2) download data via this data portal, and (3) use API data access (under development). On this page, you can find information about how to access the data in these different ways, how the data is organized, what is currently available, and how to open, extract, and analyze the data.

Getting Started with Portal Data Access

  1. Create an account

    Register for a DREAMS account. New accounts remain inactive until an administrator manually approves access. We try to do this quickly, but please also plan on at least one business day to let us approve.

  2. Generate your API key

    Once approved, users generate a unique API key from the account page. That key is then supplied in the X-API-Key header when downloading files. API keys help us make our data as open as possible to the community without the burden of repetitive and unnecessary bot traffic.

  3. Browse and download

    Browse and download the data. Example download commands are available through the portal, and will eventually be possible through DREAMS_Python.

Getting Started with Globus Data Access

All public data is available via Globus. Globus data access is facilitated via this link. No account is required, and no further permission is required to access and use that data, though appropriate attribution, including citations to the DREAMS collaboration, is expected.

How DREAMS data is organized in the portal

Before describing the DREAMS data organization, it is useful to discuss the nature of DREAMS data itself. DREAMS simulation suites are built from large numbers of simulations, organized within a simulation campaign, that vary both the initial conditions and multiple physical or numerical parameters. Critically, the overarching goal of the DREAMS project is to probe the nature of Dark Matter using this simulation-based approach. As such, any simulation within DREAMS is characterized by (i) a dark matter model, (ii) a physical galaxy formation model, and (iii) a strategy for sampling or varying input parameters. Then, within each simulation campaign, we have (i) a large list of simulations, typically around 1,024, each with a unique initial condition and associated set of parameters, and (ii) raw and post-processed simulation products including snapshot files, structure catalogs, merger trees, and more.

As such, the data within this DREAMS data portal is sorted using the following hierarchy:

Currently, the portal is populated with CDM and WDM models. This is expected to grow in the future to include more complex variations on the initial matter power spectrum, such as ETHOS h-peak/k-peak models, and variations on late-time Dark Matter dynamics, such as self-interacting dark matter or atomic dark matter.

We have four general classes of simulation suites under consideration, with two of those classes already released. We have completed, published, and released simulations using full volumes and Milky Way halo-mass zoom-ins. We are currently executing, and will soon release, simulation suites targeting low-mass dwarf halos and simulation suites targeting a wide range of halo masses, the so-called mass-varied suites.

Model Type Suite Cosmology Simulations Status
Cold dark matter (CDM) Boxes IllustrisTNG Varied 1024 Complete
Milky Way IllustrisTNG Varied 1024 Complete
Dwarfs Ramses Varied 512 Running
Dwarfs FIRE Varied 512 Running
Warm dark matter (WDM) Boxes N-body Fixed 600 Complete
Boxes N-body Varied 1000 Complete
Boxes IllustrisTNG Varied 1024 Complete
Milky Way IllustrisTNG Fixed 1024 Complete
Hpeak Kpeak Boxes N-body Fixed 400 Complete
Boxes N-body Varied 1024 Planned
Boxes IllustrisTNG Varied 1024 Planned
ETHOS Milky Way N-body Varied 1024 Planned
Milky Way IllustrisTNG Varied 1024 Planned
Atomic dark matter (ADM) Milky Way N-body Varied 1024 Planned
Milky Way IllustrisTNG Varied 1024 Planned
Snapshot / redshift reference 91 outputs ยท 15 retained full snapshots

The full FoF/Subfind catalog series is available for every output from 000 through 090. Full retained snapshot files are currently available only for a subset of those outputs, as confirmed from the mounted DREAMS data directories.

Snapshot Scale Factor Redshift Availability
000 0.063 14.99 Retained snapshot + FoF/Subfind
001 0.070 13.34 FoF/Subfind only
002 0.077 11.98 Retained snapshot + FoF/Subfind
003 0.082 11.20 FoF/Subfind only
004 0.087 10.48 FoF/Subfind only
005 0.094 9.69 FoF/Subfind only
006 0.100 9.00 Retained snapshot + FoF/Subfind
007 0.105 8.49 FoF/Subfind only
008 0.111 8.01 FoF/Subfind only
009 0.116 7.60 FoF/Subfind only
010 0.121 7.24 FoF/Subfind only
011 0.127 6.89 Retained snapshot + FoF/Subfind
012 0.132 6.56 FoF/Subfind only
013 0.137 6.28 FoF/Subfind only
014 0.143 6.01 Retained snapshot + FoF/Subfind
015 0.148 5.75 FoF/Subfind only
016 0.154 5.50 FoF/Subfind only
017 0.161 5.23 FoF/Subfind only
018 0.167 5.00 Retained snapshot + FoF/Subfind
019 0.172 4.80 FoF/Subfind only
020 0.178 4.61 FoF/Subfind only
021 0.183 4.45 FoF/Subfind only
022 0.189 4.30 FoF/Subfind only
023 0.194 4.15 FoF/Subfind only
024 0.200 4.01 Retained snapshot + FoF/Subfind
025 0.205 3.87 FoF/Subfind only
026 0.211 3.73 FoF/Subfind only
027 0.216 3.62 FoF/Subfind only
028 0.223 3.49 FoF/Subfind only
029 0.229 3.36 FoF/Subfind only
030 0.236 3.24 FoF/Subfind only
031 0.242 3.12 FoF/Subfind only
032 0.249 3.01 Retained snapshot + FoF/Subfind
033 0.257 2.90 FoF/Subfind only
034 0.263 2.80 FoF/Subfind only
035 0.269 2.72 FoF/Subfind only
036 0.276 2.63 FoF/Subfind only
037 0.282 2.54 FoF/Subfind only
038 0.289 2.46 FoF/Subfind only
039 0.296 2.38 FoF/Subfind only
040 0.303 2.30 FoF/Subfind only
041 0.310 2.22 FoF/Subfind only
042 0.318 2.15 FoF/Subfind only
043 0.325 2.07 FoF/Subfind only
044 0.333 2.00 Retained snapshot + FoF/Subfind
045 0.341 1.93 FoF/Subfind only
046 0.349 1.86 FoF/Subfind only
047 0.358 1.80 FoF/Subfind only
048 0.366 1.73 FoF/Subfind only
049 0.375 1.67 FoF/Subfind only
050 0.384 1.60 FoF/Subfind only
051 0.393 1.54 FoF/Subfind only
052 0.403 1.48 Retained snapshot + FoF/Subfind
053 0.412 1.43 FoF/Subfind only
054 0.422 1.37 FoF/Subfind only
055 0.434 1.30 FoF/Subfind only
056 0.443 1.26 FoF/Subfind only
057 0.455 1.20 FoF/Subfind only
058 0.466 1.14 FoF/Subfind only
059 0.478 1.09 FoF/Subfind only
060 0.489 1.05 FoF/Subfind only
061 0.501 1.00 Retained snapshot + FoF/Subfind
062 0.513 0.95 FoF/Subfind only
063 0.525 0.90 FoF/Subfind only
064 0.538 0.86 FoF/Subfind only
065 0.550 0.82 FoF/Subfind only
066 0.564 0.77 FoF/Subfind only
067 0.577 0.73 Retained snapshot + FoF/Subfind
068 0.591 0.69 FoF/Subfind only
069 0.605 0.65 FoF/Subfind only
070 0.620 0.61 FoF/Subfind only
071 0.635 0.58 FoF/Subfind only
072 0.650 0.54 FoF/Subfind only
073 0.665 0.50 Retained snapshot + FoF/Subfind
074 0.681 0.47 FoF/Subfind only
075 0.698 0.43 FoF/Subfind only
076 0.714 0.40 FoF/Subfind only
077 0.731 0.37 FoF/Subfind only
078 0.749 0.34 FoF/Subfind only
079 0.771 0.30 FoF/Subfind only
080 0.789 0.27 FoF/Subfind only
081 0.808 0.24 Retained snapshot + FoF/Subfind
082 0.827 0.21 FoF/Subfind only
083 0.847 0.18 FoF/Subfind only
084 0.867 0.15 FoF/Subfind only
085 0.888 0.13 FoF/Subfind only
086 0.910 0.10 FoF/Subfind only
087 0.931 0.07 FoF/Subfind only
088 0.954 0.05 FoF/Subfind only
089 0.977 0.02 FoF/Subfind only
090 1.000 0.00 Retained snapshot + FoF/Subfind

To date, only TNG simulation suites have been publicly released. However, models using GIZMO/FIRE, RAMSES, and ChaNGa are currently being executed. As those simulation suites are completed and published, they will be populated into this same space.

Generally, each simulation campaign includes 1,024 simulations, i.e. 1,024 boxes within each simulation campaign. Since each simulation uses a unique initial condition coupled to a unique set of adopted physical and numerical parameters, this large number of simulations provides sufficiently large datasets to train emulators that can predict properties of the resulting galaxy population with reasonable accuracy.

Further DREAMS documentation -- ReadTheDocs

Goals

High-level scientific goals: constraining dark matter and understanding the small-scale physics of the Universe.

General Description

Organization of DREAMS simulations by model, type, and suite, including snapshot/redshift reference material.

Dark Matter Models

Background on the cold, warm, ETHOS, Hpeak-Kpeak, and atomic dark matter model families used across DREAMS.

Simulation Types

Overview of cosmological boxes, Milky Way zoom-ins, and dwarf zoom-ins.

Simulation Codes

Descriptions of the IllustrisTNG, Ramses, and N-body branches of the project.

Parameters

Parameter-space documentation for the varied astrophysical and cosmological controls in DREAMS suites.