Skip to main content

Detecting extreme values ​​in Python

Detecting extrema in a signal.​

Generating a spurious signal as an example.​

  • 3 [Hz] signal + 0.01 [Hz] signal + noise
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from scipy import signal

tmp

Examining the properties of the signal.​

def acorr(df: pd.DataFrame, ra: int = 3, fs=100):
   x = np.correlate(df.Data.values, df.Data.values, mode='full')
   t = (np.arange(len(x)) - len(x) / 2 + 0.5) / fs
   x /= x.max()
   fig, ax = plt.subplots()
   ax.set_xlim(-ra, ra)
   ax.set_ylim(0, 1)
   ax.plot(t, x)
   ax.set_title('Autocorrelation')
   ax.grid(axis='x')
   ax.set_xticks(np.linspace(-ra, ra, 2 * ra + 1))
   plt.show()

acorr(df)

The main signal is 1/3 [Hz]. Therefore, a low-pass filter with a cutoff of 1/3 [Hz] is applied.

tmp

Applying a low-pass filter and detecting extrema.

If the extrema are not correctly detected even after applying a low-pass filter at 1/3 [Hz], the frequency is lowered.

tmp

fs = 10
lpf = 0.1
b, a = signal.butter(5, lpf / fs * 2, 'low')
df['Filter'] = signal.filtfilt(b, a, df.Data.values)
max_idx = signal.argrelextrema(df['Filter'].values, np.greater)[0]
min_idx = signal.argrelextrema(df['Filter'].values, np.less)[0]

df['max_index'] = False
df.iloc[max_idx[0], 2] = True
df['min_index'] = False
df.iloc[min_idx[0], 3] = True

fig, ax = plt.subplots()
df.plot(ax=ax)
ax.set_xlim(['00:00:00', '00:00:30'])
ax.scatter(
   df.loc[df['max_index'], ['Filter']].index, 
   df.loc[df['max_index'], ['Filter']], 
   color='tab:orange', 
   zorder=3)
ax.scatter(
   df.loc[df['min_index'], ['Filter']].index, 
   df.loc[df['min_index'], ['Filter']], 
   color='tab:green', 
   zorder=3)
plt.show()

Handling 2D arrays in C

It is convenient to manage memory where numbers are stored by summarizing the number of rows, number of columns, and the memory itself in a structure.

#include <stdio.h>
#include <stdlib.h>

typedef struct {
 float *data;
 int col_size;
 int row_size;
} Mat;

void MatInit(Mat *mat, int row_size, int col_size) {
 mat->row_size = row_size;
 mat->col_size = col_size;
 mat->data = (float *)calloc(row_size * col_size, sizeof(float));
}

float *MatAt(Mat *mat, int i, int j) {
 return mat->data + i * mat->col_size + j;
}

void MatFree(Mat *mat) { free(mat->data); }

int main(void) {
 Mat mat;
 MatInit(&mat, 30, 5); // Initialize a matrix with 30 rows and 5 columns
 *MatAt(&mat, 0, 0) = 50; // Assign 50 to row 0, column 0
 printf("%f\n", *MatAt(&mat, 0, 0)); // Print the value at row 0, column 0

 MatFree(&mat);

 return 0;
}
Tags:

Implementation of NMF (HALS)

import numpy as np

X = np.array([[1, 1], [2, 1], [3, 1.2], [4, 1], [5, 0.8], [6, 1]])

n_components, n_samples, n_features, = (2,) + X.shape
W = np.random.uniform(size = (n_samples, n_components))
H = np.random.uniform(size = (n_components, n_features))

eps = 1e-4

# NMF
for i in range(100):
   # update B
   A = X.T.dot(W)
   B = W.T.dot(W)
   for j in range(n_components):
       tmp = H[j, :] + A[:, j] - H.T.dot(B[:, j])
       H[j, :] = np.maximum(tmp, eps)

   # update A
   C = X.dot(H.T)
   D = H.dot(H.T)
   for j in range(n_components):
       tmp = W[:, j] * D[j, j] + C[:, j] - W.dot(D[:, j])
       W[:, j] = np.maximum(tmp, eps)
       norm = np.linalg.norm(W[:, j])
       if norm > 0:
           W[:, j] /= norm

print(W)
print(H)
print(W.dot(H))

Setting up a DNS server in WSL2

Disabling automatic generation of /etc/resolv.conf​

Edit /etc/wsl.conf as follows:

[network]
generateResolvConf = false

Creating /etc/resolv.conf​

For example, if the DNS server is 1.1.1.1, edit /etc/resolv.conf as follows:

nameserver 1.1.1.1

Preventing Deletion​

/etc/resolv.conf is deleted when WSL2 is restarted. To prevent this:

sudo chattr +i /etc/resolv.conf

Reference

When nokogiri cannot be installed under brew

The following two errors occurred:

zlib is missing; necessary for building libxml2
xslt is missing. Please locate mkmf.log to investigate how it is failing.

Solution

  • Install libxslt and libxml2
  • Specify the path of libxml2 installed by brew
brew install libxslt libxml2
bundle config build.nokogiri --use-system-libraries --with-xml2-include=$(brew --prefix libxml2)/include/libxml2

Reference sites

Windows OpenSSH Permission denied issue

  • Password login is possible (although disabled in the settings)
  • Permission denied occurs when using public key authentication

When connecting to localhost:22, the following error occurs:

hikari@localhost: Permission denied (publickey,keyboard-interactive).

image

Cause

It appears that the Administrators group, i.e., "administrator users", is referencing the public key in C:\ProgramData\ssh\administrators_authorized_keys for authentication by default.

Change this to $env:userprofile\.ssh\authorized_keys.

Solution

Open C:\ProgramData\ssh\sshd_config with administrator privileges and comment out the following two lines:

image

- Match Group administrators
-       AuthorizedKeysFile __PROGRAMDATA__/ssh/administrators_authorized_keys
+ #Match Group administrators
+ #      AuthorizedKeysFile __PROGRAMDATA__/ssh/administrators_authorized_keys

After saving, restart the service.

Restart-Service sshd

Installing Powermline on Termux

pkg install golang

Powermline Installation

This is almost the same as the Powermline installation method

go get -u github.com/justjanne/powermline-go

Setting in .profile

Open ~/ .profile and add the following:

GOPATH=$HOME/go
function _update_ps1() {
   PS1="$( $GOPATH/bin/powermline-go -newline -error $? )"
}
if [ "$TERM" != "linux" ] && [ -f "$GOPATH/bin/powermline-go" ]; then
   PROMPT_COMMAND="_update_ps1; $PROMPT_COMMAND"
fi

Font Setup

For example, install yuru7/PlemolJP.

wget https://github.com/yuru7/PlemolJP/releases/download/v0.4.0/PlemolJP_NF_v0.4.0.zip
unzip PlemolJP_NF_v0.4.0.zip
cp PlemolJP_NF_v0.4.0/PlemolJP35Console_NF/PlemolJP35ConsoleNF-Medium.ttf $HOME/.term
ux/font.ttf
rm PlemolJP_NF_v0.4.0 -rf
rm PlemolJP_NF_v0.4.0.zip

SSH to termux

pkg install openssh

If an error occurs​

If you run this on the Android version of termux, an error will occur. We recommend deleting the existing termux and installing termux from Github.

Start the server

sshd

Configure the public key

Add the client's public key (~/ .ssh/id_*.pub) to ~/ .ssh/authorized_keys in termux.

It's easy to copy and paste the public key by sending it to yourself via email or DM on Twitter.

# Example:
echo ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKYztjZfIVMl5o0J2DrigTsl1XgbSKMUgYCpfOfhMtmw hikari@B450M-K >> ~/.ssh/authorized_keys

Login

# Check the IP address of your smartphone
ip a

ssh 192.168.x.x -p 8022

Disable Password Authentication in OpenSSH on Windows

Open an elevated terminal​

You need administrator privileges to edit the configuration file, so open a terminal with administrator privileges.

Right-click on the terminal icon and select "Run as administrator".

image

image

Open the configuration file in the terminal​

Run the following command:

notepad C:\ProgramData\ssh\sshd_config

image

Edit the configuration file​

- # PasswordAuthentication yes
+ PasswordAuthentication no

image

Change it to

image

and save the changes.

Restart the SSH server​

Return to the terminal and run the following command to restart the SSH server:

Restart-Service sshd

Connection test​

Test if the configuration is enabled.

Run the following command and if you see:

ssh localhost
user@localhost: Permission denied (publickey,keyboaard-interactive).

Then it's OK.