James Jun

single-thread, no-gpu

------------------------------
Summary of ~/ceph/groundtruth/hybrid_synth/static_siprobe/rec_64c_1200s_11/irc2/raw_geom.prm
------------------------------
Recording format
    Recording file:         ~/ceph/groundtruth/hybrid_synth/static_siprobe/rec_64c_1200s_11/raw.mda
    Probe file:             ~/ceph/groundtruth/hybrid_synth/static_siprobe/rec_64c_1200s_11/geom.csv
    Recording Duration:     1200.0s
    Data Type:              int16
    #Channels in file:      64
    #Sites:                 64
    #Shanks:                1
Pre-processing
    Filter type:            bandpass
    Filter range (Hz):      [300.0, 8000.0]
    Common ref:             trimmean
    Whiten:                 0
    FFT threshold:          8
    blank threshold:        0
Events
    #Spikes:                510735
    Feature extracted:      gpca
    #Sites/event:           18
    maxDist_site_um:        50
    maxDist_site_spk_um:    100
    spkLim_ms:              [-0.250, 0.750]
    #Features/event:        45
    #PC/chan:               2.500000e+00
Cluster
    #Clusters:              88
    #Unique events:         508992
    min. spk/clu:           30
    Cluster method:         drift-knn
    knn:                    30
    step_sec_drift:         20.0s
    batch_sec_drift:        300.0s
Auto-merge
    delta_cut:              1.000
    maxWavCor:              0.990
Runtime (s)
    Detect + feature (s):   156.2s
    Cluster runtime (s):    36.5s
    merge runtime (s):      12.4s
    Total runtime (s):      205.0s
    Runtime speed:          x5.9 realtime
    Processing speed:       2491.4 spikes/s
memory usage (GiB):         -0.018
    detect:                 -0.048
    sort:                   -0.018
    auto-merge:             -0.018
Execution
    irc2 version:           v5.5.4
    fGpu (GPU use):         0
    fParfor (parfor use):   0
    fLargeRecording:        0
    Parameter file:         ~/ceph/groundtruth/hybrid_synth/static_siprobe/rec_64c_1200s_11/irc2/raw_geom.prm
------------------------------

single-thread, gpu

------------------------------
Summary of ~/ceph/groundtruth/hybrid_synth/static_siprobe/rec_64c_1200s_11/irc2/raw_geom.prm
------------------------------
Recording format
    Recording file:         ~/ceph/groundtruth/hybrid_synth/static_siprobe/rec_64c_1200s_11/raw.mda
    Probe file:             ~/ceph/groundtruth/hybrid_synth/static_siprobe/rec_64c_1200s_11/geom.csv
    Recording Duration:     1200.0s
    Data Type:              int16
    #Channels in file:      64
    #Sites:                 64
    #Shanks:                1
Pre-processing
    Filter type:            bandpass
    Filter range (Hz):      [300.0, 8000.0]
    Common ref:             trimmean
    Whiten:                 0
    FFT threshold:          8
    blank threshold:        0
Events
    #Spikes:                510737
    Feature extracted:      gpca
    #Sites/event:           18
    maxDist_site_um:        50
    maxDist_site_spk_um:    100
    spkLim_ms:              [-0.250, 0.750]
    #Features/event:        45
    #PC/chan:               2.500000e+00
Cluster
    #Clusters:              84
    #Unique events:         502857
    min. spk/clu:           30
    Cluster method:         drift-knn
    knn:                    30
    step_sec_drift:         20.0s
    batch_sec_drift:        300.0s
Auto-merge
    delta_cut:              1.000
    maxWavCor:              0.990
Runtime (s)
    Detect + feature (s):   92.5s
    Cluster runtime (s):    20.3s
    merge runtime (s):      12.7s
    Total runtime (s):      125.4s
    Runtime speed:          x9.6 realtime
    Processing speed:       4072.3 spikes/s
memory usage (GiB):         0.049
    detect:                 0.049
    sort:                   0.049
    auto-merge:             0.049
Execution
    irc2 version:           v5.5.4
    fGpu (GPU use):         1
    fParfor (parfor use):   0
    fLargeRecording:        0
    Parameter file:         ~/ceph/groundtruth/hybrid_synth/static_siprobe/rec_64c_1200s_11/irc2/raw_geom.prm
------------------------------

12-thread, no-GPU

------------------------------
Summary of ~/ceph/groundtruth/hybrid_synth/static_siprobe/rec_64c_1200s_11/irc2/raw_geom.prm
------------------------------
Recording format
    Recording file:         ~/ceph/groundtruth/hybrid_synth/static_siprobe/rec_64c_1200s_11/raw.mda
    Probe file:             ~/ceph/groundtruth/hybrid_synth/static_siprobe/rec_64c_1200s_11/geom.csv
    Recording Duration:     1200.0s
    Data Type:              int16
    #Channels in file:      64
    #Sites:                 64
    #Shanks:                1
Pre-processing
    Filter type:            bandpass
    Filter range (Hz):      [300.0, 8000.0]
    Common ref:             trimmean
    Whiten:                 0
    FFT threshold:          8
    blank threshold:        0
Events
    #Spikes:                510735
    Feature extracted:      gpca
    #Sites/event:           18
    maxDist_site_um:        50
    maxDist_site_spk_um:    100
    spkLim_ms:              [-0.250, 0.750]
    #Features/event:        45
    #PC/chan:               2.500000e+00
Cluster
    #Clusters:              88
    #Unique events:         508992
    min. spk/clu:           30
    Cluster method:         drift-knn
    knn:                    30
    step_sec_drift:         20.0s
    batch_sec_drift:        300.0s
Auto-merge
    delta_cut:              1.000
    maxWavCor:              0.990
Runtime (s)
    Detect + feature (s):   58.3s
    Cluster runtime (s):    24.8s
    merge runtime (s):      9.4s
    Total runtime (s):      92.5s
    Runtime speed:          x13.0 realtime
    Processing speed:       5522.3 spikes/s
memory usage (GiB):         0.065
    detect:                 0.065
    sort:                   0.018
    auto-merge:             0.018
Execution
    irc2 version:           v5.5.4
    fGpu (GPU use):         0
    fParfor (parfor use):   1
    fLargeRecording:        0
    Parameter file:         ~/ceph/groundtruth/hybrid_synth/static_siprobe/rec_64c_1200s_11/irc2/raw_geom.prm
------------------------------

12-thread, GPU

------------------------------
Summary of ~/ceph/groundtruth/hybrid_synth/static_siprobe/rec_64c_1200s_11/irc2/raw_geom.prm
------------------------------
Recording format
    Recording file:         ~/ceph/groundtruth/hybrid_synth/static_siprobe/rec_64c_1200s_11/raw.mda
    Probe file:             ~/ceph/groundtruth/hybrid_synth/static_siprobe/rec_64c_1200s_11/geom.csv
    Recording Duration:     1200.0s
    Data Type:              int16
    #Channels in file:      64
    #Sites:                 64
    #Shanks:                1
Pre-processing
    Filter type:            bandpass
    Filter range (Hz):      [300.0, 8000.0]
    Common ref:             trimmean
    Whiten:                 0
    FFT threshold:          8
    blank threshold:        0
Events
    #Spikes:                510737
    Feature extracted:      gpca
    #Sites/event:           18
    maxDist_site_um:        50
    maxDist_site_spk_um:    100
    spkLim_ms:              [-0.250, 0.750]
    #Features/event:        45
    #PC/chan:               2.500000e+00
Cluster
    #Clusters:              84
    #Unique events:         502857
    min. spk/clu:           30
    Cluster method:         drift-knn
    knn:                    30
    step_sec_drift:         20.0s
    batch_sec_drift:        300.0s
Auto-merge
    delta_cut:              1.000
    maxWavCor:              0.990
Runtime (s)
    Detect + feature (s):   38.1s
    Cluster runtime (s):    15.2s
    merge runtime (s):      9.3s
    Total runtime (s):      62.6s
    Runtime speed:          x19.2 realtime
    Processing speed:       8152.4 spikes/s
memory usage (GiB):         0.018
    detect:                 0.018
    sort:                   -0.029
    auto-merge:             -0.029
Execution
    irc2 version:           v5.5.4
    fGpu (GPU use):         1
    fParfor (parfor use):   1
    fLargeRecording:        0
    Parameter file:         ~/ceph/groundtruth/hybrid_synth/static_siprobe/rec_64c_1200s_11/irc2/raw_geom.prm
------------------------------

Computing resource

OS: CentOS Linux 7
Memory: 251.7 GiB
Processor: Intel® Xeon(R) CPU E5-2643 v4 @ 3.40GHz × 24 
Graphics: Quadro P4000/PCIe/SSE2

after filling the missing values with the median value per dimension do a low-rank approximation (svd) and re-project the matrix and repeat until the missing value vector values converge. It was tested to about ~70% missing matrix.

also, he runs a fast-tsne (Mannas’ implementation) followed by K-means to cluster the genetic data. Better discrimination when you apply missing-value filling procedure using the low-rank matrix method.

https://www.dell.com/community/Inspiron/Suspend-resume-problems-on-Ubuntu-18-04/td-p/6072410

1. I already had the Nvidia drivers installed and was already using them
2. sudo systemctl disable nvidia-fallback.service
3. Edited /etc/default/grub and added the line :
    GRUB_CMDLINE_LINUX=”nouveau.blacklist=1″
4. sudo update-grub
5. Restarted the laptop

wget http://developer.download.nvidia.com/compute/cuda/10.1/Prod/local_installers/cuda_10.1.243_418.87.00_linux.run
sudo sh cuda_10.1.243_418.87.00_linux.run

QT is required for yakuake

sudo apt-get install build-essential

sudo apt-get install qtcreator

sudo apt-get install qt5-default

sudo apt-get install qt5-doc

sudo apt-get install qt5-doc-html qtbase5-doc-html

sudo apt-get install qtbase5-examples

[installing yakuake (pull-down terimnal)]
sudo apt-get install yakuake

install docker
run sudo service docker start

share docker daemon on windows and wsl

Export and import WSL environment:
wsl –export Ubuntu-18.04 ubuntu.tar

sudo apt update
sudo apt install snapd 
sudo snap install code-insiders --classic

add alias code='code-insiders in .bashrc

Also Installed npm (permission), yarn, docker and Matlab 2019b