Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev/igormat/ci jenkins #2

Merged
merged 3 commits into from
Sep 13, 2018
Merged

Dev/igormat/ci jenkins #2

merged 3 commits into from
Sep 13, 2018

Conversation

mlx3im
Copy link

@mlx3im mlx3im commented Sep 13, 2018

this changes required for adding spdk to swx_ci project

@mike-dubman mike-dubman merged commit 786fd6c into master Sep 13, 2018
@mike-dubman mike-dubman deleted the dev/igormat/ci_jenkins branch September 13, 2018 11:32
@sashakot
Copy link

sashakot commented Sep 17, 2018

Based on https://github.com/Mellanox/swx_ci

cd spdk
/hpc/noarch/git_projects/swx_ci/template/template/test_jenkins.sh build_gcc

should run CI test from command line

Actually it doesn't' work

sashakot@dev-r-vrt-088 ~/hpc-work/spdk (master) $ /hpc/noarch/git_projects/swx_ci/template/template/test_jenkins.sh build_gcc
-bash: /hpc/noarch/git_projects/swx_ci/template/template/test_jenkins.sh: No such file or directory

test_jenkins.sh was moved

@sashakot
Copy link

In case of spdk, a prerequisites should be installed before running configure

./scripts/pkgdep.sh

How that can be done with in with ci ?

@sashakot
Copy link

How additional arguments can be passed to ?

#
# Prepare build environment
#
prepare_test() {
    echo " ==== Prepare ===="
    mkdir -p $build_test_dir
    case $build_mode in
            "cmake" )
                    cd $build_test_dir
                    cmake -DCMAKE_INSTALL_PREFIX=$(pwd) -DWITH_LINT=ON -DBUILD_DOC=OFF -DCMAKE_BUILD_TYPE="Release" $sourcedir
                    ;;
            "autotools" )
                    #todo: support out of source tree build
                    (cd $sourcedir; ./configure --prefix=$build_test_dir)
                    ;;
    esac
}

@sashakot
Copy link

on_script_init()
{
    if [ -f $sourcedir/CMakeLists.txt ]; then
            build_mode="cmake"
    elif [ -f $sourcedir/Makefile.am ]; then
            build_mode="autotools"
            ./autogen.sh
    elif [ -f $sourcedir/Makefile ]; then
            build_mode="autotools"
    else
            build_mode="unsupported"
            echo Unsupported build system - please update $0
            exit 1
    fi

    if [ -f $sourcedir/.gitmodules ]; then
            git submodule init ||:
            git submodule update ||:
    fi
}

That's is not correct in general case. Not all projects have autogen.sh

aviadye pushed a commit that referenced this pull request Jun 17, 2019
OCF queue list needs to be managed synchronously.
This patch uses our own mutex to achieve that because we cannot rely on
  ocf_mngt_cache_lock() as it may produce deadlocks when using
  cleaner.

Alternative way would be to use trylock, but
  we need to register a poller for it and do locking concurently
  which doesn't seem to be possible in callbacks of io channel.

We agreed with OCF team that we are going to change this part
  when OCF will deliver safe ocf_mngt_cache_lock() function.

This patch fixes a very rare failure on our CI that looked like this:
```
04:33:02 vbdev_ocf.c: 134:stop_vbdev: *NOTICE*: Not stopping cache instance 'Malloc0' because it is referenced by other OCF bdev
04:33:03 MalCache1: Core core2 successfully removed
04:33:03 MalCache1: Stopping cache
04:33:03 MalCache1: Done saving cache state!
04:33:03 src/ocf/mngt/ocf_mngt_cache.c:1576:2: runtime error: pointer index expression with base 0x000000000000 overflowed to 0xffffffffffffffa8
04:33:03     #0 0x7f3c52d54c26 in _ocf_mngt_cache_stop src/ocf/mngt/ocf_mngt_cache.c:1576
04:33:03     #1 0x7f3c52d5579f in ocf_mngt_cache_stop src/ocf/mngt/ocf_mngt_cache.c:1657
04:33:03     #2 0x7f3c52cbe9f4 in stop_vbdev /var/jenkins/workspace/NVMe_tests/nvme_phy_autotest/spdk/lib/bdev/ocf/vbdev_ocf.c:147
04:33:03     #3 0x7f3c52cbf4a0 in vbdev_ocf_destruct /var/jenkins/workspace/NVMe_tests/nvme_phy_autotest/spdk/lib/bdev/ocf/vbdev_ocf.c:216
```

Change-Id: Id6fafb444958f3becdc480e44762074c6c081e1f
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450682
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
AlekseyMarchuk pushed a commit that referenced this pull request Jul 2, 2019
ASAN needs to be LD_PRELOADed before SPDK fio_plugin
in order to analyze its code.
Just adding that will report any issues in fio binary as
well as the fio_plugin.
To prevent known fio leaks from affecting the results,
a suppression list for LeakSanitizer (used in conjunction with ASAN).

At this time the suppression list contains known leaks
for fio 3.3. The list might need adjustments as fio
version is updated.

Side note. Even though it is possible to specify directory
to ignore ("leak:/usr/src/fio/"). Which in theory should
suppress any leaks in fio. It has side effect of hiding
SPDK leaks as well, since the fio_plugins leaks are
seen as coming from /usr/src/fio/ioengines.c.

See below for examples of each suppressed error:

Direct leak of 42 byte(s) in 4 object(s) allocated from:
    #0 0x7f9d52f3e320 in strdup (/lib64/libasan.so.5+0x3b320)
    #1 0x41f267 in get_new_job /usr/src/fio/init.c:490

Direct leak of 914936 byte(s) in 10397 object(s) allocated from:
    #0 0x7f74422e8ea6 in __interceptor_calloc (/lib64/libasan.so.5+0x10dea6)
    #1 0x46402e in log_io_piece /usr/src/fio/iolog.c:214

Direct leak of 608 byte(s) in 19 object(s) allocated from:
    #0 0x7f74422e8ca8 in __interceptor_malloc (/lib64/libasan.so.5+0x10dca8)
    #1 0x44c4e1 in add_to_dump_list /usr/src/fio/parse.c:1039
    #2 0x44c4e1 in parse_option /usr/src/fio/parse.c:1098

Direct leak of 173 byte(s) in 20 object(s) allocated from:
    #0 0x7f744227153d in strdup (/lib64/libasan.so.5+0x9653d)
    #1 0x44b50d in __handle_option /usr/src/fio/parse.c:718

Indirect leak of 111925528 byte(s) in 1271881 object(s) allocated from:
    #0 0x7f74422e8ea6 in __interceptor_calloc (/lib64/libasan.so.5+0x10dea6)
    #1 0x46402e in log_io_piece /usr/src/fio/iolog.c:214

Indirect leak of 171 byte(s) in 19 object(s) allocated from:
    #0 0x7f744227153d in strdup (/lib64/libasan.so.5+0x9653d)
    #1 0x44c4ed in add_to_dump_list /usr/src/fio/parse.c:1040
    #2 0x44c4ed in parse_option /usr/src/fio/parse.c:1098

Indirect leak of 167 byte(s) in 19 object(s) allocated from:
    #0 0x7f744227153d in strdup (/lib64/libasan.so.5+0x9653d)
    #1 0x44c502 in add_to_dump_list /usr/src/fio/parse.c:1042
    #2 0x44c502 in parse_option /usr/src/fio/parse.c:1098

Change-Id: I9b5811993508421be50b12af160645c77ea93d7e
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456315
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
AlekseyMarchuk pushed a commit that referenced this pull request Jul 2, 2019
There are some leaks in libiscsi that is external to SPDK.
With this patch, any leaks in libiscsi will be suppressed.

Please see examples of the leaks right now when running
bdevperf in initiator.sh test:

Direct leak of 176 byte(s) in 1 object(s) allocated from:
    #0 0x7fd64dddfe50 in calloc (/lib64/libasan.so.5+0xeee50)
    #1 0x7fd64c60edb4 in scsi_cdb_inquiry (/usr/lib64/iscsi/libiscsi.so.8+0x13db4)

Indirect leak of 256 byte(s) in 1 object(s) allocated from:
    #0 0x7fd64dddfe50 in calloc (/lib64/libasan.so.5+0xeee50)
    #1 0x7fd64c603a84  (/usr/lib64/iscsi/libiscsi.so.8+0x8a84)

Indirect leak of 52 byte(s) in 1 object(s) allocated from:
    #0 0x7fd64dddfe50 in calloc (/lib64/libasan.so.5+0xeee50)
    #1 0x7fd64c60d755 in scsi_malloc (/usr/lib64/iscsi/libiscsi.so.8+0x12755)
    #2 0x7fd64c6097f8  (/usr/lib64/iscsi/libiscsi.so.8+0xe7f8)

Change-Id: I80ef23b1464841f683344c7aab99f1658a46cd36
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456766
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
allen-mlnx pushed a commit that referenced this pull request Aug 18, 2020
Not all JSON methods require 'params' field to be supplied.
Verification of the JSON is done on server side in
parse_single_request().

We should not attempt to process garbage values on correct
JSON config file during app start.

Segfault can be observed if following valid JSON config is supplied:
{
	"method": "framework_wait_init"
}
Resulting in:
json_config.c:388:13: runtime error: applying non-zero offset 18446744073709551600 to null pointer
AddressSanitizer:DEADLYSIGNAL
=================================================================
==3386067==ERROR: AddressSanitizer: SEGV on unknown address (pc 0x0000007260ff bp 0x7ffe6ea06890 sp 0x7ffe6ea067e0 T0)
==3386067==The signal is caused by a READ memory access.
==3386067==Hint: this fault was caused by a dereference of a high value address (see register values below).  Dissassemble the provided pc to learn which register was used.
    #0 0x7260ff in app_json_config_load_subsystem_config_entry /home/tzawadzk/spdk/lib/event/json_config.c:391
    #1 0x7cbb13 in msg_queue_run_batch /home/tzawadzk/spdk/lib/thread/thread.c:505
    #2 0x7cd00a in thread_poll /home/tzawadzk/spdk/lib/thread/thread.c:581
    #3 0x7cfe18 in spdk_thread_poll /home/tzawadzk/spdk/lib/thread/thread.c:689
    #4 0x71d6ef in _reactor_run /home/tzawadzk/spdk/lib/event/reactor.c:326
    #5 0x71eb00 in reactor_run /home/tzawadzk/spdk/lib/event/reactor.c:382
    #6 0x71f911 in spdk_reactors_start /home/tzawadzk/spdk/lib/event/reactor.c:477
    #7 0x718237 in spdk_app_start /home/tzawadzk/spdk/lib/event/app.c:691
    #8 0x407e94 in main /home/tzawadzk/spdk/app/spdk_tgt/spdk_tgt.c:120
    #9 0x7f0f2eef2041 in __libc_start_main ../csu/libc-start.c:308
    #10 0x4079ad in _start (/home/tzawadzk/spdk/build/bin/spdk_tgt+0x4079ad)

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I7ef1a764467817ad788fdf5dbe17eaeb99dcc22e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3256
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
EugeneKochetov pushed a commit that referenced this pull request May 29, 2022
The controller data structure may be freed before subsystem resume done
callback, we can take endpoint as the input parameter to avoid this issue.

AddressSanitizer: heap-use-after-free on address 0x625000046100 at pc 0x00000082818f bp 0x7fff7b09bd10 sp 0x7fff7b09bd00
READ of size 8 at 0x625000046100 thread T0 (reactor_0)
    #0 0x82818e in vfio_user_dev_quiesce_resume_done /spdk/lib/nvmf/vfio_user.c:2147
    #1 0x782cc0 in subsystem_state_change_done /spdk/lib/nvmf/subsystem.c:634
    #2 0xad047b in _call_completion /spdk/lib/thread/thread.c:2344
    #3 0xabc48d in msg_queue_run_batch /spdk/lib/thread/thread.c:710
    #4 0xac0670 in thread_poll /spdk/lib/thread/thread.c:926
    #5 0xac0ead in spdk_thread_poll /spdk/lib/thread/thread.c:986
    #6 0x9a5b4f in _reactor_run /spdk/lib/event/reactor.c:920
    #7 0x9a6442 in reactor_run /spdk/lib/event/reactor.c:958
    #8 0x9a717c in spdk_reactors_start /spdk/lib/event/reactor.c:1060
    #9 0x99884a in spdk_app_start /spdk/lib/event/app.c:643
    #10 0x407e82 in main /spdk/app/nvmf_tgt/nvmf_main.c:75
    #11 0x7f822095ff42 in __libc_start_main (/lib64/libc.so.6+0x23f42)
    #12 0x407abd in _start (/spdk/build/bin/nvmf_tgt+0x407abd)

0x625000046100 is located 0 bytes inside of 8320-byte region [0x625000046100,0x625000048180)
freed by thread T0 (reactor_0) here:
    #0 0x7f82219ff91f in __interceptor_free (/lib64/libasan.so.5+0x10d91f)
    #1 0x837059 in _free_ctrlr /spdk/lib/nvmf/vfio_user.c:2976
    #2 0x837327 in free_ctrlr /spdk/lib/nvmf/vfio_user.c:2996
    #3 0x843541 in nvmf_vfio_user_close_qpair /spdk/lib/nvmf/vfio_user.c:3742
    #4 0x7d1d91 in nvmf_transport_qpair_fini /spdk/lib/nvmf/transport.c:604
    #5 0x7ad922 in _nvmf_qpair_destroy /spdk/lib/nvmf/nvmf.c:1055
    #6 0x761362 in nvmf_qpair_request_cleanup /spdk/lib/nvmf/ctrlr.c:4026
    #7 0x761906 in spdk_nvmf_request_free /spdk/lib/nvmf/ctrlr.c:4041
    #8 0x75a931 in nvmf_qpair_free_aer /spdk/lib/nvmf/ctrlr.c:3576
    #9 0x7ae626 in spdk_nvmf_qpair_disconnect /spdk/lib/nvmf/nvmf.c:1127
    #10 0x83db36 in _vfio_user_qpair_disconnect /spdk/lib/nvmf/vfio_user.c:3433
    #11 0xabc48d in msg_queue_run_batch /spdk/lib/thread/thread.c:710
    #12 0xac0670 in thread_poll /spdk/lib/thread/thread.c:926
    #13 0xac0ead in spdk_thread_poll /spdk/lib/thread/thread.c:986
    #14 0x9a5b4f in _reactor_run /spdk/lib/event/reactor.c:920
    #15 0x9a6442 in reactor_run /spdk/lib/event/reactor.c:958
    #16 0x9a717c in spdk_reactors_start /spdk/lib/event/reactor.c:1060
    #17 0x99884a in spdk_app_start /spdk/lib/event/app.c:643
    #18 0x407e82 in main /spdk/app/nvmf_tgt/nvmf_main.c:75
    #19 0x7f822095ff42 in __libc_start_main (/lib64/libc.so.6+0x23f42)

previously allocated by thread T0 (reactor_0) here:
    #0 0x7f82219fff16 in __interceptor_calloc (/lib64/libasan.so.5+0x10df16)
    #1 0x837413 in nvmf_vfio_user_create_ctrlr /spdk/lib/nvmf/vfio_user.c:3010
    #2 0x83bc68 in nvmf_vfio_user_accept /spdk/lib/nvmf/vfio_user.c:3313
    #3 0xabfbd8 in thread_execute_timed_poller /spdk/lib/thread/thread.c:872
    #4 0xac0c75 in thread_poll /spdk/lib/thread/thread.c:960
    #5 0xac0ead in spdk_thread_poll /spdk/lib/thread/thread.c:986
    #6 0x9a5b4f in _reactor_run /spdk/lib/event/reactor.c:920
    #7 0x9a6442 in reactor_run /spdk/lib/event/reactor.c:958
    #8 0x9a717c in spdk_reactors_start /spdk/lib/event/reactor.c:1060
    #9 0x99884a in spdk_app_start /spdk/lib/event/app.c:643
    #10 0x407e82 in main /spdk/app/nvmf_tgt/nvmf_main.c:75
    #11 0x7f822095ff42 in __libc_start_main (/lib64/libc.so.6+0x23f42)

SUMMARY: AddressSanitizer: heap-use-after-free /spdk/lib/nvmf/vfio_user.c:2147 in vfio_user_dev_quiesce_resume_done

Change-Id: Icf5e5b360b9107a3c5eb960ae59b7fe10ace1c66
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11420
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
AlekseyMarchuk pushed a commit that referenced this pull request May 17, 2023
- fix precision
 when one convert to seconds and then multiply
 we can have precision errors
 for example if one have 77ms, it will go to 0 when converted to seconds
 and then multiply that 0 by 1000 will return 0 instead of 77ms.

- fix mismatch nsec/usec
 nsec was multiplied by 1000*1000 while usec by 1000*1000*1000
 it should be the opposite.
 anyway the implementation had changed.

- implementation description
* env_ticks_to_msec: j / (tick_hz / 1000)
  this is exactly the same as (j * 1000) / tick_hz (eq #2).
  but this implementation (eq #2) can only handle 54b in j (before overflowing)
  because of the multiplication by 1000 (10b).
  with the correct implementation we use all 64b in j.
  we assume that tick_hz will be prefectly divisible by 1000 so we are ok.

* env_ticks_to_usec: j / (tick_hz / (1000 * 1000))
  same as in msec case, we use all 64b in j.
  here we assume that tick_hz is perfectly divisible by (1000 * 1000)
  i.e. we assume that CPU frequency is some multiple of 1MHz.

* env_ticks_to_nsec: (j * 1000) / (tick_hz / (1000 * 1000))
  in this case we can't assume that tick_hz is divisible by 10^9
  because there are many CPUs with 2.8GHz or 3.3GHz for example.
  so we multiply j by 1000
  this means that we can only handle correctly j up to 54b.
  (64b - 10b, 10b for the *1000 operation)

Signed-off-by: Amir Haroush <amir.haroush@huawei.com>
Signed-off-by: Shai Fultheim <shai.fultheim@huawei.com>
Change-Id: Ia8ea7f88b718df206fa0731e3f39f419ee922aa7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17078
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
AlekseyMarchuk pushed a commit that referenced this pull request May 17, 2023
This test is not really useful. The intended purposes
I think were:

1) Validate that the height of the tree is never too
   high.
2) Capture the time to insert a bdev's names into the
   rb tree.

But #1 isn't being tested correctly.  It only tests
the height of a name that was *just* inserted.  RB
trees always insert a new node at the leaf, followed
by some rotations as necessary to maintain the RB
tree properties.

#2 doesn't help with anything.  The NOTICELOGs don't
get printed, and even if they did, we aren't validating
them at all.

So just remove this test altogether.  We don't need
to validate the RB tree code itself, and we shouldn't
try to validate performance with a unit test.  The way
the test is constructed is way too expensive too - it
takes up the vast majority of the time to run the
entire unit test suite. Removing just this one test cut
the time for unittest.sh from 28 seconds to less than
3 seconds on my system.

Fixes issue spdk#3013.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If2253d4927ca5f1184fb165ed3f2c25befe9f4a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17980
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
yshestakov pushed a commit that referenced this pull request Sep 19, 2023
Ubsan with clang complains when using spdk_ioviter with more iters than
declared in the array:

  iov.c:69:9: runtime error: index 3 out of bounds for type 'struct spdk_single_ioviter[2]'
  #0 0x5df709 in spdk_ioviter_firstv /home/vagrant/spdk_repo/spdk/lib/util/iov.c:69:9
  #1 0x53780b in raid5f_xor_stripe /home/vagrant/spdk_repo/spdk/module/bdev/raid/raid5f.c:270:24
  #2 0x531bd8 in raid5f_submit_write_request /home/vagrant/spdk_repo/spdk/module/bdev/raid/raid5f.c:520:2
  #3 0x52a03a in raid5f_submit_rw_request /home/vagrant/spdk_repo/spdk/module/bdev/raid/raid5f.c:596:9
  #4 0x548c17 in test_raid5f_write_request /home/vagrant/spdk_repo/spdk/test/unit/lib/bdev/raid/raid5f.c/raid5f_ut.c:550:2
  #5 0x544e18 in test_raid5f_submit_rw_request /home/vagrant/spdk_repo/spdk/test/unit/lib/bdev/raid/raid5f.c/raid5f_ut.c:714:3
  #6 0x553e61 in __test_raid5f_submit_full_stripe_write_request /home/vagrant/spdk_repo/spdk/test/unit/lib/bdev/raid/raid5f.c/raid5f_ut.c:878:3
  #7 0x543f84 in run_for_each_raid5f_config /home/vagrant/spdk_repo/spdk/test/unit/lib/bdev/raid/raid5f.c/raid5f_ut.c:748:3
  #8 0x527ac1 in test_raid5f_submit_full_stripe_write_request /home/vagrant/spdk_repo/spdk/test/unit/lib/bdev/raid/raid5f.c/raid5f_ut.c:885:2
  #9 0x7f4a71a0960a  (/usr/lib64/libcunit.so.1+0x460a) (BuildId: 9c82dd336cbccd99721651ac0a04435e746e0fc0)
  #10 0x7f4a71a09937  (/usr/lib64/libcunit.so.1+0x4937) (BuildId: 9c82dd336cbccd99721651ac0a04435e746e0fc0)
  #11 0x7f4a71a0a897 in CU_run_all_tests (/usr/lib64/libcunit.so.1+0x5897) (BuildId: 9c82dd336cbccd99721651ac0a04435e746e0fc0)
  #12 0x524fe8 in main /home/vagrant/spdk_repo/spdk/test/unit/lib/bdev/raid/raid5f.c/raid5f_ut.c:1006:2
  #13 0x7f4a711d750f in __libc_start_call_main (/usr/lib64/libc.so.6+0x2750f) (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
  #14 0x7f4a711d75c8 in __libc_start_main@GLIBC_2.2.5 (/usr/lib64/libc.so.6+0x275c8) (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
  #15 0x4235b4 in _start (/home/vagrant/spdk_repo/spdk/test/unit/lib/bdev/raid/raid5f.c/raid5f_ut+0x4235b4) (BuildId: 028d075edd1a7cd17881fd678ef076adfdbac13d)

Fix this by making iters a zero-length array and put it in a union with a
two-element array to keep the default size for compatibility.

Change-Id: I8573b015755e9986cdadbfa1705d269d51a7c2b7
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18402
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
EugeneKochetov pushed a commit that referenced this pull request Jan 18, 2024
Do not depend on start of the application
to free the I/O buffer in closing poller.
Instead do that in main on every exit path.

There is a negative test case with hello_sock,
which shows mem leak when ASAN is enabled.
See test case:
test/iscsi_tgt/sock/sock.sh:126

Direct leak of 1024 byte(s) in 1 object(s) allocated from:
    #0 0x7f48846ba097 in calloc (/usr/lib64/libasan.so.8+0xba097)
    #1 0x408ffe in main /home/vagrant/spdk_repo/spdk/examples/sock/hello_world/hello_sock.c:562
    #2 0x7f488370750f in __libc_start_call_main (/usr/lib64/libc.so.6+0x2750f)

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib7dcad385e5c01128cab71d108832a8ac0594709
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/20734
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
EugeneKochetov pushed a commit that referenced this pull request Jan 18, 2024
String allocated in avahi_string_list_to_string()
was not freed after it was no longer needed.

Fixes spdk#3122

Direct leak of 102 byte(s) in 2 object(s) allocated from:
    #0 0x7fc255cd92ef in malloc (/usr/lib64/libasan.so.8+0xd92ef) (BuildId: 6f17f87dc4c1aa9f9dde7c4856604c3a25ba4872)
    #1 0x7fc255bf5fb4 in avahi_malloc (/usr/lib64/libavahi-common.so.3+0x3fb4) (BuildId: e99b431894489a77a31921493f198df4850090e0)
    #2 0x7fc255bf6e74 in avahi_string_list_to_string (/usr/lib64/libavahi-common.so.3+0x4e74) (BuildId: e99b431894489a77a31921493f198df4850090e0)
    #3 0x4a3c93 in mdns_resolve_callback /home/tzawadzk/spdk/module/bdev/nvme/bdev_mdns_client.c:261
	....

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib0eae278d59152c97572531bef59efef36600f13
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/20849
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
EugeneKochetov pushed a commit that referenced this pull request Jan 18, 2024
zcopy_phase is used in the hot path hence move to first cache line.

dif_enabled is not used by generic code but used in the hot path on
tcp and rdma transport.

By using single bits for flags we can reduce 8 bytes and create some
headroom for future flags.

Change-Id: Ib3a1059bb47b9bce18db902099e6680ff5277148
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21345
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
EugeneKochetov pushed a commit that referenced this pull request Apr 9, 2024
As per typedef in nvme.h the spdk_nvme_cpl argument should be a
pointer to a const struct.

This fixes runtimer error under clang >= 17.x which now makes the
-fsanitize=function available for C and which on our end is being
enabled via -fsanitize=undefined under UBSAN.

Error in question:

 Test: test_spdk_nvme_detach ...passed
  Test: test_nvme_completion_poll_cb ...passed
  Test: test_nvme_user_copy_cmd_complete
.../root/spdk/lib/nvme/nvme.c:417:2: runtime error: call to function
dummy_cb through pointer to incorrect function type 'void (*)(void *,
const struct spdk_nvme_cpl *)'
/root/spdk/test/unit/lib/nvme/nvme.c/nvme_ut.c:584: note: dummy_cb
defined here
    #0 0x5098e0 in nvme_user_copy_cmd_complete
       /root/spdk/lib/nvme/nvme.c:417:2
    #1 0x532161 in test_nvme_user_copy_cmd_complete
       /root/spdk/test/unit/lib/nvme/nvme.c/nvme_ut.c:604:2
    #2 0x7f08c952266a  (/usr/lib64/libcunit.so.1+0x466a) (BuildId:
       d99e3b60795f2ce01ada820b4b7e3cd84d8150fe)
    #3 0x7f08c95229c7  (/usr/lib64/libcunit.so.1+0x49c7) (BuildId:
       d99e3b60795f2ce01ada820b4b7e3cd84d8150fe)
    #4 0x7f08c9523a9f in CU_run_all_tests
       (/usr/lib64/libcunit.so.1+0x5a9f) (BuildId:
d99e3b60795f2ce01ada820b4b7e3cd84d8150fe)
    #5 0x55555e in run_tests /root/spdk/lib/ut/ut.c:169:3
    #6 0x552aec in spdk_ut_run_tests /root/spdk/lib/ut/ut.c:225:8
    #7 0x522d52 in main
       /root/spdk/test/unit/lib/nvme/nvme.c/nvme_ut.c:1664:17
    #8 0x7f08c8c28149 in __libc_start_call_main
       (/usr/lib64/libc.so.6+0x28149) (BuildId:
7ea8d85df0e89b90c63ac7ed2b3578b2e7728756)
    #9 0x7f08c8c2820a in __libc_start_main@GLIBC_2.2.5
       (/usr/lib64/libc.so.6+0x2820a) (BuildId:
7ea8d85df0e89b90c63ac7ed2b3578b2e7728756)
    #10 0x42b6a4 in _start
	(/root/spdk/test/unit/lib/nvme/nvme.c/nvme_ut+0x42b6a4)
(BuildId: 6fc2caaf777030becad2d0f660ec68443f3380b4)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior
/root/spdk/lib/nvme/nvme.c:417:2 in
./test/unit/unittest.sh: line 85: 75349 Aborted                 (core
dumped) $valgrind $testdir/lib/nvme/nvme.c/nvme_ut

Change-Id: Iddbd5fc0dee0ef6a6cc1f032e079f6119e76aed9
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22025
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
EugeneKochetov pushed a commit that referenced this pull request Apr 9, 2024
This will only print the specified error message once per second.

Some caveats:

1) This uses spdk_get_ticks() and friends which is technically a circular
   dependency since env_dpdk depends on log. log.h doesn't include env.h,
   and I'm not sure we should add it due to this dependency. It means caller
   needs to have env.h included before using this new macro.
2) The ratelimit is global, so if thread A prints it as time 0s, and thread
   B prints it at time 0.5s, the thread B one will get squashed. I think this
   is the desired behavior.
3) Similar to #2, it means there is small race where two threads do the
   ERRLOG at the exact same time after the 1 second has expired. In this case
   both will read the expired value and do the print. Again, I think this is
   fine, let's not overcomplicate this.
4) This doesn't take argument values into account. So if this macro is getting
   called many many times but each time with different parameters (i.e. a
   controller name) then the log will only show one of them.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I642350f39bacaaa79358308511dab511e6570cae
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22615
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
EugeneKochetov pushed a commit that referenced this pull request Aug 21, 2024
The bdev_null_create RPC set rpc_construct_null structure and copied
it to null_bdev_opts structure. However, this was not efficient or
clean.

The difference of rpc_construct_null structure and null_bdev_opts
structure were:
1. null_bdev_opts::block_size was block size,
   rpc_construct_null::block_size was data block size
2. null_bdev_opts::uuid was pointer, rpc_construct_null::uuid was
   instance

For #1, as we do for malloc bdev, we can use null_bdev_opts::block_size
as data block size. If null_bdev_opts::block_size is data block size,
we do not have to compare opts::block_size and opts::md_size.

For #2, as we do for malloc bdev, null_bdev_opts::uuid can be instance
and we can copy null_bdev_opts::uuid to spdk_bdev::uuid.

Additionally, spdk_null_bdev_opts is not a public data structure. Hence,
it should not have "spdk_" prefix.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: If53702071cf02fe23d03f464a63c2495c70e1f12
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24109
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants