forked from westerndigitalcorporation/libzbc
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathREADME
385 lines (286 loc) · 14.4 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
Copyright (C) 2009-2014, HGST, Inc.
I. Introduction
===============
I.1. Overview
-------------
libzbc is a simple library providing functions for manipulating disks
supporting the Zoned Block Command (ZBC) and Zoned-device ATA command set
(ZAC) disks. libzbc implemention is compliant with the latest drafts of
the ZBC and ZAC standards defined by INCITS technical committee T10 and
T13 (respectively).
In addition to supporting ZBC and ZAC disks, libzbc also implements an
emulation mode allowing emulating the behavior of a host-managed zoned
disks using a regular file or a raw standard block device as backing
storage.
Several example applications using libzbc are available under the tools
directory.
I.2. Standards Versions Supported
---------------------------------
The "master" code branch provides libzbc v2.0.0 which currently supports
the ZBC standard draft Rev 02 (released on 2014/11/11) and Rev 01 of the
ZAC standard draft (released on 2014/12/09) completed with the "ZAC Zone
State Machine et aliae" document f14183r0 published on 2014/12/23.
The "r04" code branch implements libzbc v4.0.0 and provides support for
the revision 4 of ZBC and ZAC standards.
Both the ZBC and ZAC standards are still being developped. Changes to the
command set and command interface can be expected before the final public
release. Any change in the standards will be incorporated into libzbc as
disks become available.
I.3. License
------------
libzbc is distributed under the terms of the of the BSD 2-clause license
("Simplified BSD License" or "FreeBSD License"). A copy of this license
with the library copyright can be found in the COPYING.BSD file.
With the exception of the linear key value store (lkvs) tool, all example
applications under the tools directory are distributed under the terms of
the GNU Lesser General Public License version 3, or any later version. A
copy of version 3 this license can be found in the COPYING.LESSER file.
The lkvs tool is distributed under the terms of the of the GNU General
Public License version 3, or any later version. A copy of version 3 of
this license can be found in the COPYING.GPL file.
libzbc and all its example applications are distributed "as is,"
without technical support, and WITHOUT ANY WARRANTY, without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Along with libzbc, you should have received a copy of the BSD 2-clause
license, GNU General Public License and og the GNU Lesser General Public
License. If not, please see <http://opensource.org/licenses/BSD-2-Clause>
and <http://www.gnu.org/licenses/>.
I.4. Contact and Bug Reports
----------------------------
Please contact Damien Le Moal (damien.lemoal@hgst.com) and
Adam Manzanares (adam.manzanares@hgst.com) to report problems.
II. Compilation and installation
================================
II.1. Requiremensts
-------------------
The compilation of libzbc requires that the autoconf, GTK3 and GTK3
development packages be installed on the target host. The GTK3 packages
are only necessary for the gzbc GUI example application. If this
application is not compiled, installing the GTK3 and GTK3 developemnt
packages is not necessary.
II.2. Compilation
-----------------
To compile the library and all example applications, execute the
following commands.
> sh ./autogen.sh
> ./configure
> make
II.3. Installation
------------------
To install the library and all example applications, as root, execute
the following command.
> make install
The library files are installed in /usr/lib. The library header files
are installed in /usr/include/libzbc The executable files for the example
applications are installed in /usr/bin.
II.4. Compilation for device tests
----------------------------------
The test directory contains several test programs and scripts allowing
testing the compatibility of libzbc with a particular device. That is,
testing if a device follows the same standard as currently supported by
libzbc (see section I.2).
The compilation of these test programs is disabled by default. To compile
the test programs, libzbc must be configured as follows.
> sh ./autogen.sh
> ./configure --with-test
> make
And as root,
> make install
This last command does not copy the test programs and scripts anywhere.
All defined tests can be executed directly form the test directory by
executing the zbc_test.sh script.
> cd test
> sudo ./zbc_test.sh /dev/<SG node>
Each test executed outputs a log file in the test/log directory. These
files can be consulted in case of failed test to identify the reason
for the test failure.
III. Usage
==========
III.1 Kernel Version
--------------------
libzbc functions operate using device handles which are obtained by
executing the zbc_open function. This function argument can be a regular
file or a legacy raw block device file for operating in emulation mode, or
the SG node device file (/dev/sg<x>) of a ZBC disk or a ZAC disk. Kernel
support is required for these SG nodes to exist.
Linux kernel introduced support for ZBC device type with version 3.17. Any
kernel with version 3.17 and above will thus recognize SAS ZBC devices
and create an SG node for these disks, but will not create a regular
block device (such as /dev/sd<y>).
The device signature of SATA ZAC disks is not recognized by kernels
before 3.19. For these kernels, no SG device node is created, resulting
in the impossibility to access ZAC disks. To enable ZAC device signature
handling for kernels between version 3.17 and 3.19, the following patches
must be used.
https://git.kernel.org/cgit/linux/kernel/git/hare/scsi-devel.git/commit/?h=zac.v2&id=ec07b9a8b168499c6009bf5bfe925a0a10b49729
https://git.kernel.org/cgit/linux/kernel/git/hare/scsi-devel.git/commit/?h=zac.v2&id=02582e1667f05b0c64f2998877216e395e7fda52
https://git.kernel.org/cgit/linux/kernel/git/hare/scsi-devel.git/commit/?h=zac.v2&id=70eb02d6eca2c0706b388a3971f9e156bac092d7
III.2 Library Functions
-----------------------
libzbc exports the following functions.
+------------------------------+------------------------------------+
| Function | Description |
+==============================+====================================+
| zbc_open | Open a ZBC device |
+------------------------------+------------------------------------+
| zbc_close | Close a ZBC device |
+------------------------------+------------------------------------+
| zbc_get_device_info | Get sector and size information |
+------------------------------+------------------------------------+
| zbc_report_zones | Get zone information |
+------------------------------+------------------------------------+
| zbc_report_nr_zones | Get the number of zones |
+------------------------------+------------------------------------+
| zbc_list_zones | Get device zone information |
+------------------------------+------------------------------------+
| zbc_open_zones | Explicitely open a zone |
+------------------------------+------------------------------------+
| zbc_close_zones | Close an open zone |
+------------------------------+------------------------------------+
| zbc_finish_zones | Finish a zone |
+------------------------------+------------------------------------+
| zbc_reset_write_pointer | Reset a zone write pointer |
+------------------------------+------------------------------------+
| zbc_pread | Read data from a zone |
+------------------------------+------------------------------------+
| zbc_pwrite | Write data to a zone |
+------------------------------+------------------------------------+
| zbc_write | Write data to a sequential zone |
+------------------------------+------------------------------------+
| zbc_flush | Flush data to disk |
+------------------------------+------------------------------------+
The current implementation of these functions is NOT thread safe. In
particular, concurrent write operations by multiple threads to the
same zone may result in write errors without write ordering control
by the application.
Additionally, the following functions are also provided to facilitate
application development and tests.
+------------------------+------------------------------------------+
| Function | Description |
+==============================+====================================+
| zbc_set_log_level | Set log level of the library functions |
+------------------------+------------------------------------------+
| zbc_disk_type_str | Get a string description of a disk type |
+------------------------+------------------------------------------+
| zbc_disk_model_str | Get a string description of a disk model |
+------------------------+------------------------------------------+
| zbc_zone_type_str | Get a string description of a zone type |
+------------------------+------------------------------------------+
| zbc_zone_condition_str | Get a string description of a zone state |
+------------------------+------------------------------------------+
More detailed information on these functions is available through the
comments in the file include/libzbc/zbc.h.
III.3 Native Mode Operation
---------------------------
The standard Linux kernel does not create a blocks device file for
host-managed ZBC and ZAC disks. As a result, these devices are only
accessible through their associated SG node (/dev/sgx device file).
Opening a native ZBC or ZAC disk with libzbc thus needs to be done by
specifying the device SG node. Further accesses (report zones,
reset zone write pointer, etc) can then proceed using the device
handle provided by the zbc_open function.
III.4 Emulation Mode Operation
------------------------------
libzbc can emulate host-managed disks operation using a regular
file or a raw standard block device (regular disk or loopback device).
The use of the library in such case is identical to the native mode
case, assuming that the emulated device is first configured by executing
the zbc_set_zones tool (see next section).
IV. Example Applications
========================
Under the tools directory, several simple applications are available
as examples. These appliations are as follows.
IV.1. gzbc (tools/gui)
----------------------
gzbc provides a graphical user interface showing zone information of a
zoned device. It also displays the write status (write pointer position)
of zones graphically using color coding (red for written space and green
for unwritten space). Some operations on zones can also be executed
directly from the interface (reset zone write pointer, open zone, close
zone, etc).
IV.2. zbc_report_zones (tools/report_zones/)
--------------------------------------------
This application illustrates the use of the zone reporting functions
(zbc_report_zones, zbc_report_nr_zones, zbc_list_zones). It obtains
the zone information of a device and displays it in readable form on
the standard output.
IV.3. zbc_open_zone (tools/open_zone/)
--------------------------------------------------
This application illustrates the use of the zbc_open_zone
function allowing opening a zone.
IV.4. zbc_close_zone (tools/close_zone/)
--------------------------------------------------
This application illustrates the use of the zbc_close_zone
function allowing closing a zone.
IV.5. zbc_finish_zone (tools/finish_zone/)
--------------------------------------------------
This application illustrates the use of the zbc_finish_zone
function allowing finishing a zone.
IV.6. zbc_reset_write_ptr (tools/reset_write_ptr/)
--------------------------------------------------
This application illustrates the use of the zbc_reset_write_pointer
function allowing resetting the write pointer of a zone to the start
LBA of the zone.
IV.7. zbc_read_zone (tools/read_zone/)
--------------------------------------
This application reads data from a zone, up to the zone write pointer
location and either send the read data to the standard output or copy
the data to a regular file. It implementation uses the function zbc_pread.
IV.8. zbc_write_zone (tools/write_zone/)
----------------------------------------
This application illustrates the use of the zbc_pwrite function which
write data to a zone at the zone write pointer location.
IV.9. zbc_set_zones (tools/set_zones/)
--------------------------------------
This application can be used to initialize the ZBC emulation mode for
a regular file or a raw standard block device.
IV.10. zbc_set_write_ptr (tools/set_write_ptr/)
----------------------------------------------
This application can be used to set the write pointer of a zone of an
emulated ZBC device to any LBA value (within the range of the specified
zone). It is intended for testing purposes only and is not valid for
native ZBC devices.
IV.11. lkvs (tools/lkvs/)
------------------------
### Purpose
ZBC drives have three types of zones: conventional, sequential preferred,
and sequential only. This project aims to create a key/value store on
top of a drive that contains seqential only zones exclusively. I.e. all writes
to the drive must be linear within the LBA space of one zone.
This work is intended to be a demonstration of how to build systems software on
top of a ZBC drive. This is not a full featured key/value store and we do not
intend it to be so.
The main limitations of the current design is that we require all
of the metadata for the KV store to fit into main memory and there is no garbage
collection built into LKVS. Therfor, updates to an existing key are not
supported.
If you want to build the python or java bindings please pass the option
of --with-pybind or --with-javabind to the configure script, to build the
corresponding bindings.
### How To Run LKVS tests
Must be done after make install of libzbc.
```
cd tools/lkvs
sudo make check-local LKVSDEVFILE=/path/to/zbcdevice
```
Also make sure that src/test/zbc_reset.sh is executable.
### How to Build Python or Java bindings
```
cd tools/lkvs
make pybind
make javabind
```
Cleaning the bindings
```
make pybind-clean
make javabind-clean
```
### Using LKVS
#### C++
See src/examples
#### Python
See src/pybind/example.py
#### Java
See src/javabind/Tester.java
### LKVS Contact Information
Contact: adam.manzanares@hgst.com