Unix Tips

Tue Apr 09 2019

Some random typical unix user tips

Vim

Vim - install vim with ruby support

sudo apt-get install vim-nox;

vim command-t error: command-t.vim could not load the C extension
ruby version…
expected version …

SOSO SOLUTION : if already installed brew install vim, simply go to command-t plugin dir ~/.vim/bundle/command-t and run: rake make

BEST SOLUTION : use CtrlP plugin instead

vim remove ^M at end of lines:

vim split window vertical horizontal down vote

Ctrl+W, S #(lower case) for horizontal splitting
Ctrl+W, V #(lower case) for vertical splitting
Ctrl+W, Q #to close one
Ctrl+W, Ctrl+W #to switch between windows
Ctrl+W, J (xor K, H, L) #to switch to adjacent window (intuitively up, down, left, right)

Vim - get Vim version

go into vim and type :ve:

:ve[rsion]

Vim - Increment number under cursor

Vim In normal mode, typing Ctrl-A will increment the next number, and typing Ctrl-X will decrement the next number

Vim - Change method first letter case : when upper case to lower case

.,$s/function \(\w\)/function \L\1/g
.,$s/::\(\w\)/::\L\1/g this would be better : .,$s/::\(\u\)\(\l\)/::\L\1\2/g to avoid selecting ::EH
.,$s/->\(\w\)/->\L\1/g
.,$s/::\v(\l)(\u\u)/::\U\1\2/g this is used to correct changing a constant into first letter lower case

Unix

ps -p 2344 -o comm=
sudo su
su <username>
sudo passwd <username>

echo $SHELL   # /bin/bash

Unix - see where filesystem is mounted

df

Unix - recover overwritten file

grep -i -a -B100 -A100 'text in the deleted file' /dev/sda1

Where /dev/sda1 should be the mount point of the hard drive containing the delted file (find it out with df)

Thanks to se

Unix - Check if package is installed

dpkg -l {package_name}
dpkg -l | less

Unix - Set an environmental variable this is useful to load different configurations based on that

SetEnv SPECIAL_PATH /foo/bin

Unix - SSH

ssh and execute command

ssh <user>@<host> -p 2222 -t "cd /var/www/my-project && composer update; bash"

Unix - Change permissions for write

sudo chown -R `whoami`:`id -gn` /var/www/html

Unix - connect to linux from unix ssh

ssh <username>@<hostname_or_ip> -p 2222

Unix - copy file from local to remote

sftp <host_user>@<hostname_or_ip>

Unix - Move all files and dirs inside dir to some dir inside that same dir

mv ./!(Webservice) ./Webservice/

Unix - Append contents of a file to end of other file

# use single > if you want to replace the entire file
RUN cat /tmp/id_rsa.pub >> ~/.ssh/authorized_keys

Unix - Allow user create dir if not owner or in group

To allow a user to create directory if he is not the owner nor in the group of the parent dir : If don’t want to change owner nor group, then add the user to the parent dir’s group, and give the group permissions to write: How to add user to an existing group

sudo usermod -a -G [group-name] [user-name]

IMPORTANT -a -G ARE ESSENTIAL because otherwise the user is removed from all other groups (-a: append)

Unix - What I did to move all files from a dir to its parent dir was

cd to/the/dir
mv * ../

Unix - Replace exact pattern in files

greprep -roTBf 'BSR' 'Bsr' .

Unix - Delete files

find . -name ‘*.greprep.tmp’ -delete

Unix - Count Number of files in dir

ls -l . | egrep -c '^-'

Unix - Show directory elements sorted by nubmer

ls -1vla .

Unix - Show directory elements sorted by date

ls -lt .

Unix - debian - Add existing user to group

addgroup <username> <groupname>

Unix - debian - Add group with specific GID

addgroup -g <GID> <groupname>

Unix - sed replace in file

Search for REVERSE_DOMAIN string in /some/dir/docker-compose.yml and replace it in place with the value contained in the variable $reversedomain.

sed -i -e "s/REVERSE_DOMAIN/$reversedomain/g" "/some/dir/docker-compose.yml"

Sed error illegal byte sequence

Unix - get OS version

uname -a
# Linux 901b171682e9 4.15.0-20-generic #21-Ubuntu SMP Tue Apr 24 06:16:15 UTC 2018 x86_64 Linux

Git

git server setup

Git development and production environments: commit -> dev update, push -> prod update

mkdir ~/proj
cd ~/proj
git init --bare

Git - Pushing code to two remotes

git remote add production https://usrpath@bitbucket.org/usrpath/<project_name>.git
git remote set-url --add --push production ssh://<username>@<hostname_or_ip>:2222/home/<username>/gitrepos/<project_name>.git
git remote set-url --add --push production https://usrpath@bitbucket.org/usrpath/<project_name>.git

git remote rm production
git remote -v

Git - Remove already committed folder, that was added to gitignore

git rm -r --cached some-directory
git commit -m 'Remove the now ignored directory "some-directory"'
git push origin master

git remote add origin https://usrpath@bitbucket.org/usrpath/bsrdb.git

How to remove local untracked files from the current Git branch

Undo uncommitted

Undo changes

How to get back to the latest commit after checking out a previous commit?

Git - How to forget about tracked files but now added to .gitignore

git rm --cached <file>

Git - Delete untracked files

git clean -f

Git - Undo `git add .`

un-stage (rollback git add .)

git rm --cached .

Git - Script to inspect logs using git

sudo su
grep -R "185.10.224.66" /var/log/apache2 > /home/<username>/logs/mylog.txt
chown <username> /home/<username>/logs/mylog.txt
chgrp users /home/<username>/logs/mylog.txt
exit

cd /home/<username>/logs
git add .
git commit -m “made a new request”

sudo su
cd /var/log/apache2
grep -R "185.10.224.66" . > /home/<username>/logs/mylog.txt
chown <username> /home/<username>/logs/mylog.txt
chgrp users /home/<username>/logs/mylog.txt
exit

cd /home/<username>/logs
git diff

How to unstage a git file

PHP

PHP - Deleting all files in a dir

foreach (glob('/path/to/dir/*') as $filepath) if (is_file($filepath)) unlink($filepath);

PHP - ini Ubuntu

PHP ini in ubuntu

add bootstrap file to phpunit

PHP - Composer

Require latest version dev git composer

Get php version from script

To update only this single package:

composer.phar update doctrine/doctrine-fixtures-bundle

Apache

Apache - serve files outside document root

Apache Serve Files outside of document root

Alias /requestedDir /real/dir/folder

Apache - Ubuntu

sudo service apache2 restart

How to enable all site confs with a2ensite (while passing over 000-default.conf && default-ssl.conf)? How to enable all sites ubuntu

Apache - Homebrew Mac OSX

sudo apachectl -k start
sudo apachectl -k stop

Apache - logs

/var/log/apache2

or in Homebrew's version:

/usr/local/var/log/httpd

MsSQL Microsoft SQL server TSQL ODBC FreeTDS

MsSQL - from Mac: This is the way to go

Perfect tutorial SQL Server Mac

MsSQL - Connect with PHP PDO

$pdo = new \PDO('odbc:MYMSSQL', 'myusername', 'mypass');
$pdostmt = $pdo->prepare('SELECT TOP 10 FROM dbo.MyTableName');
if (!$pdostmt) {
    throw new Exception('There was an error');
}
if (false === $pdostmt->execute()) {
    throw new Exception('The pdo statement did not execute successfully');
}
var_dump($pdostmt->fetchAll(\PDO::FETCH_ASSOC));

This should work. Note that you do not have to pass any tag to the dsn like in : DSN=MYMSSQL => bad. Just pass the dsn name directly after odbc:MYMSSQL.

IMPORTANT: you really have to make sure you pass the correct library name. There are plenty of versions in your system in different places. I used:

In /usr/local/etc/odbcinst.ini :

[TDSODBCDriver]
Description=ODBC Driver for SQL Server
Driver=/usr/local/lib/libtdsodbc.so
UsageCount=1

In /usr/local/etc/odbc.ini :

[MYMSSQL]
Description=Some Descirption
Servername=FMyServername
Driver=TDSODBCDriver

In /usr/local/etc/freetds.conf :

# A typical Microsoft server
[FMyServername]
host = 123.456.56.789
port = 1218
tds version = 7.0
database = mydbname
dump file = /usr/local/var/log/freetds.dump
client charset = UTF-8
debug flags = 0x80

tsql -S FMyServername -U <dbusername> -P <dbpassword>

isql ODBCInfo <dbusername> <dbpassword>

If you have errors, play around with the tds version number. And note that I have a 2005 SQL Server, which should use 7.2, but it works with 7.0. So try them all if need be.

PHP PDO ODBC Connection

Unable to load dynamic library '/usr/lib/php/20160303/pdo_sqlsrv.so' - /usr/lib/php/20160303/pdo_sqlsrv.so: undefined symbol: php_pdo_register_driver in Unknown on line 0 #736

PHP PDO MSSQL SqlServer ERROR

MsSQL - install

Install pdo for microsoft sql server in ubuntu 16.04 MsSQL install ubuntu

MSSQL Ms SQL pdo constants attributes, see for example PDO::SQLSRV_ENCODING_UTF8

Connect FreeTDS ODBC on mac Home-brew

[FreeTDS][SQL Server]Unicode data in a Unicode-only collation or ntext data cannot be sent to clients using DB-Library (such as ISQL) or ODBC version 3.7 or earlier.

Solution 1 Solution 2 Solution 3

Restart apache after

MsSQL Error text PHP mssql

SELECT TOP

Select first n rows SQL

MsSQL T-SQL 402 error

Error Solution

MsSQL - Install Mac OSX

Some other tutorial

Step 1. Install PHP
```
brew tap
brew tap homebrew/core
brew install php@7.2
```
- PHP should now be in your path -- run php -v to verify that you are running the correct version of PHP. If PHP is not in your path or it is not the correct version, run the following:
```
brew link --force --overwrite php@7.2
```
Step 2. Install prerequisites
- Install the ODBC driver for macOS by following the instructions on the Linux and macOS installation page.
- In addition, you may need to install the GNU make tools:
```
brew install autoconf automake libtool
```
Step 3. Install the PHP drivers for Microsoft SQL Server
```
sudo pecl install sqlsrv
sudo pecl install pdo_sqlsrv
```

Step 4. Install Apache and configure driver loading

brew install apache2

To find the Apache configuration file for your Apache installation, run
```
apachectl -V | grep SERVER_CONFIG_FILE
```

and substitute the path for httpd.conf in the following commands:

echo "LoadModule php7_module /usr/local/opt/php@7.2/lib/httpd/modules/libphp7.so" >> /usr/local/etc/apache2/httpd.conf
(echo "<FilesMatch .php$>"; echo "SetHandler application/x-httpd-php"; echo "</FilesMatch>";) >> /usr/local/etc/apache2/httpd.conf

source

Step 5. Restart Apache and test the sample script
```
sudo apachectl —restart
```

Apache PHP multiple versions OSX

Error information:
SQLSTATE: IMSSP
Code: -49
Message: This extension requires the Microsoft ODBC Driver for SQL Server. Access the following URL to download the ODBC Driver for SQL Server for x64: https://go.microsoft.com/fwlink/?LinkId=163712

Freetds test tsql

/usr/local/opt/freetds/bin/tsql

Mac OS X

Mac OS X - Programs used and dependencies

Deleted /usr/local, and will install brew, anything inside it will be from brew

greprep is [my script] and used to search and replace files allows cleaning up too
- loc-bin : is in /home/g/Google-Drive/scipts
- depends : to be included in $PATH in .bash_profile
apache2 [osx]
- loc-bin : /System/Library/LaunchDaemons/org.apache.httpd.plist
- loc-conf/etc/apache2/httpd.conf config file
- uses : php from there in line libphpX.so
php [osx|brew] depending on version (7.2 by brew)
- depends on many modules
php7.2 [brew]
- config : ini files are in
vim [osx|brew] use brew version because osx version does not have proper ruby support for command t
- uses : command-t plugin
command-t plugin [thrid]
- uses : Vundle for install
- depends : ruby
ruby-2.3.7p456 [osx]
- loc-bin : /usr/bin/ruby
ruby-^ [rubenv]
rubenv [brew] is a tool used to manage ruby versions
- depends : autoconf, openssl, pkg-config and ruby-build
openssl [brew] openssl is keg-only, which means it was not symlinked into /usr/local, because Apple has deprecated use of OpenSSL in favor of its own TLS and crypto libraries.
brew [third]
- loc-bin : /usr/local/bin/brew
- depends : many files inside /usr/local (anything inside /usr/local was created on Home-brew install)
- cache : /Users/g/Library/Caches/Homebrew/downloads
- uninstall : remove /usr/local/opt and the cache ~/Library/Caches/Homebrew
mysql [brew]
- installation
python2.7 [osx]
unixodbc [brew]
- description : unixodbc is an ODBC driver manager. What does an ODBC driver Manager do? It basically provides the the ODBC API, such that applications can link to a single shared object to be able to talk to a range of ODBC drivers. For example in unix an application links to libodbc.so (the main driver manager shared object), without having to know at link time, which ODBC driver it is going to use. At runtime the application provides a string defining the data source it is going to be using, and this in turn defines de ODBC driver which will handle this data source. The ODBC driver manager loads the ODBC driver with dlopen(3) and passes all ODBC API calls to the driver. In this way, a ODBC application can be built without knowing which driver it will be using.
- description (odbc) : odbc is a specification for a database api. the api is independent of any dbms or operating system. the functions in the odbc api are implemented by developers of dbms-specific drivers. Applications can call these functions in these drivers to access data in a DBMS-independend manner. Anyone can write ODBC applications and drivers (Microsoft provides tools for developers to build them). The goal of ODBC is to make it possible to access any data from any application, regardless of which database management system (DBMS) is handling the data. ODBC achieves this by inserting a middle layer called a database driver between an application and the DBMS. This layer translates the application's data queries into commands that the DBMS understands
freetds [brew]
- description :
  - is a reimplementation of C libraries originally marketed by Sybase and Microsoft SQL Server. It allows open source applications such as PHP and Perl (or your own C or C++ program) to connect to Sybase or Microsoft SQL Server. FreeTDS provides drop in replacements for Microsoft DB Library, the ODBC drivers from both vendors (Sybase and Microsoft), interactive SQL and BCP utilities. TDS stands for Tabluar Data Stream, which is the protocol used to communicate with such servers. FreeTDS is distributed in C source code and should compile in any OS.
  - Keep in mind that a protocol is not an API. The server talks and recognises a protocol, anything that can send the right sequence of bytes in the right order, can communicate with it. Developers are not in the job of sending bytes, that is the job of a library. Over the years there have been many such libraries, (each with its own API) that do the work of moving SQL through a TDS pipe. ODBC, DB-Library, CT-Library all have very different api’s, but they are all one to the server, because they can speak TDS.
  - The history is that in the 1980’s when Sybase commercialized its DB server, there was no established application level protocol to transfer data between the database server and its client. That is why it developed DB-Library, which had a given API. Microsoft entered into an agreement with Sybase to commercialize its DB server. So it kept DB-Library and also developed its own library : ODBC, which had a different API. At the same time Sybase introduced a successor to DB-Library: CT-Library. All this said, the three application level libraries have their own api’s but each ends up talking to the server in the same way: with the TDS protocol.
  - The TDS protocol itself comes in different flavors and was considered a trade secret, except for the TDS 5.0 version, used exclusively by Sybase, for which documentation is available.
  - At the begginning there was only one version of TDS, but in keeping with the broad history of private ventures, they soon diverged. And they brought their own versions which cannot be interchanged.
  - History of versions: TDS 4.2 Sybase & Microsoft, 5.0 Sybase, TDS 7.0 Microsoft (for SQL Server 7.0), TDS 7.1 Microsoft (Sql Server 2000), TDS 7.2 Microsoft (SQL Server 2005), TDS 7.3 Microsoft (SQL Server 2008), TDS 7.4 Microsoft (Sql Server 2012).
  - IMPORTANT TDS 8.0 is an alias for 7.1 and TDS 9.0 for 7.2.
  - FreeTDS consists of some C libraries. Remember TDS is the protocol and FreeTDS is a set of C libraries implementing the different API’s such as DB-Library, CT-Library, ODBC. Underlying these three libraries is libtds, which handles the low level details of the TDS protocol, such as sending, receiving, and datatype conversion.
  - For our purposes we will be using ODBC flavor, since it is the api that Microsoft developed, and thus is probably more apt to talk to their database (even though as long as the library talks libtds, it should be able to communicate with Microsoft SQL Server.
  - Microsoft recommends FreeTDS to their customers who want to access Microsoft SQL Server from non Win32 clients.