Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

can be used on the mechanical hard disk? #4104

Closed
lizhichao opened this issue Mar 29, 2020 · 7 comments
Closed

can be used on the mechanical hard disk? #4104

lizhichao opened this issue Mar 29, 2020 · 7 comments
Labels
community/request Issues created by external users

Comments

@lizhichao
Copy link

If there is no SSD, yugabyte-db can be used on the mechanical hard disk?

@yugabyte-ci yugabyte-ci added the community/request Issues created by external users label Mar 29, 2020
@kmuthukk
Copy link
Collaborator

hi @lizhichao -

Unfortunately, no. YugabyteDB's design is not meant/optimized for hard disk environment. It requires SSDs.

@lizhichao
Copy link
Author

Ok

@ddorian
Copy link
Contributor

ddorian commented Mar 30, 2020

@lizhichao See: https://docs.yugabyte.com/latest/deploy/checklist/#disks
Optimizing for low latency and fast reads/inserts just can't work with HDD.
We have issues that will lower the overhead disk io:
Wal multiplex #3549
Tablet splitting #1004 (will make possible to start a table with just 1 tablet and grow from there)

After those land and by combining them with colocation + low number of tablets per-table + raid0 of some hdd, I will do some tests to see if HDD may be viable. But it will not be official though.

@tobiasmuehl
Copy link
Contributor

@ddorian Looking forward to your tests.

The design of openIO (yes, I'm aware it's just a self-hosted S3 and not a full database) uses HDD for storage and requires a small SSD to be present for metadata that is accessed more frequently. Does Yugabyte have similar optimizations? Let's say we'd run each server with 4 HDDs and 1 SSD, would the SSD be used specifically for IO-intensive tasks or are all disks load-balanced equally?

@tobiasmuehl
Copy link
Contributor

@ddorian From my understanding #1004 tablet splitting has landed. Do you think now is a "good enough" time to attempt using HDDs?

@ddorian
Copy link
Contributor

ddorian commented Apr 20, 2021

Do you think now is a "good enough" time to attempt using HDDs?

No, would need wal multiplexing too. That's more important than tablet splitting.

The design of openIO (yes, I'm aware it's just a self-hosted S3 and not a full database) uses HDD for storage and requires a small SSD to be present for metadata that is accessed more frequently. Does Yugabyte have similar optimizations? Let's say we'd run each server with 4 HDDs and 1 SSD, would the SSD be used specifically for IO-intensive tasks or are all disks load-balanced equally?

It's a completely different workload. Think of it like this: you put the metadata of openIO INSIDE yugabytedb and put yugabytedb ON TOP OF ssd.

metadata in openio ==(they are actually the same thing!)== yugabytedb-data. Putting yugabytedb in HDD is exactly like putting openIO.metadata in HDD.

@ddorian
Copy link
Contributor

ddorian commented Apr 20, 2021

As an example, seaweedfs is also an object store / s3, that is optimized for HDD. It's metadata, can reside in yugabytedb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community/request Issues created by external users
Projects
None yet
Development

No branches or pull requests

5 participants