Thursday, October 18, 2018

Installing PHP 7.2.11 in CentOS 7.4 with Pthreads enabled

Installing PHP 7.2.11 in CentOS 7.4 with pthreads enabled

This steps are meant for PHP CLI threading applications.  So web related appications are not installed along with.

OS Version: CentOS Linux release 7.4.1708 (Core)

First we need to install  PHP. Version used is php-7.2.11

1. Download php-7.2.11.tar.gz source from http://php.net/get/php-7.2.11.tar.gz/from/a/mirror
2. Extract to /usr/src
 
3. cd /usr/src/php-7.2.11
4. ./configure --with-config-file-path=/usr/local/lib --with-tsrm-pthreads --enable-pthreads=shared --enable-maintainer-zts --enable-sockets
5. make
6. make test
7. make install
8. To see the php version installed
     php -v

Now we need to install Pthreads. Source is taken from Github.

1. Download from https://github.com/krakjoe/pthreads.git
2. Extract to /usr/src/php-7.2.11/ext
3. This step may not be required, but this is how it got worked for me and i didnt tried the other way.      Rename the extracted file from pthreads-master to pthreads.
      mv /usr/src/php-7.2.11/ext/pthreads-master /usr/src/php-7.2.11/ext/pthreads
4. cd /usr/src/php-7.2.11/ext/pthreads
5. phpize
6. ./configure
7. make
8. make test
9. make install

Now we require to Rebuild our PHP installation

1. cd /usr/src/php-7.2.11
2. make clean
3. rm -f configure && ./buildconf --force
4. ./configure --with-config-file-path=/usr/local/lib --with-tsrm-pthreads --enable-pthreads=shared --enable-maintainer-zts --enable-sockets
5. make
6. make test
7. make install
   At end of make install note this path
    Installing shared extensions:     /usr/local/lib/php/extensions/no-debug-zts-20170718/

Next is the important and final step, to edit the configuration file (php.ini)
1. Give command -> php --ini
   Output ->
    Configuration File (php.ini) Path: /usr/local/lib
    Loaded Configuration File:         (none)
    Scan for additional .ini files in: (none)
    Additional .ini files parsed:      (none)

2. Copy file from /usr/src/php-7.2.11 to /usr/local/lib and rename the file to php.ini
    cp /usr/src/php-7.2.11/php.ini-development /usr/local/lib/php.ini

3. Give command -> php --ini
   Output ->
    Configuration File (php.ini) Path: /usr/local/lib
    Loaded Configuration File:         /usr/local/lib/php.ini
    Scan for additional .ini files in: (none)
    Additional .ini files parsed:      (none)

   Now the Configuration file is loaded

4. Add this line to /usr/local/lib/php.ini below
    extension=/usr/local/lib/php/extensions/no-debug-zts-20170718/pthreads.so



5. Shows the modules loaded
     php -m

    In the loaded modules you shoud be able to see the pthreads


Thursday, June 7, 2018

Steps for importing csv / text data to Postgres database table in Ubuntu / Linux

Steps for importing csv / text data to Postgres database table in Ubuntu / Linux


  • If the file is too big to process in one step, cut the file into multiple files
    • split -l 250000 filename.txt  // Will split the file into multiple files with 250000 lines
  • In case if all columns are not required to be imported, create a new file with required number of columns. Mind the delimiter used.
    • cut -d'|' -f1,3,6,33,34,35 file_name.txt >> out.txt
  • Next two lines are specific to my case
    • sed -e '/^[a-z,A_Z]/d' test.txt > test_1.txt  // To delete the lines which starts with a character.
    • sed -i '/^$/d' file.txt   //To delete empty lines
  • Copy the first line which is the column names to all the splitted file outputs, except the first file. Use "vim".
  • I used a python program to delete lines which contains number of columns not equal to three.
#Python Program
fh = open('test.txt')      # Open the Original file
f = open("test_2.txt","w") # Create a new file to store the output
for line in fh:
    countLine = line.count("|")
    if countLine == 2:
       f.write(line)
fh.close()
  • Create a table with requited columns. Better to give data type as character varying and size as larger than reqired for all columns (eg character varying (400))
  • Copy the formatted txt / csv to postgres table. Here delimiter used is '|'.
    • COPY table_name(phone_no,subscriber_name,address,pincode,place,district) FROM 'local_location/filename.txt' DELIMITER '|' CSV HEADER;
  • Possible Errors
    • Error telling some special characters present, so not able to copy.  /*  "UTF8": 0xbf   */
    • Here i have deleted the lines with that special characters. Go this website and figure out the special character.
    • https://www.utf8-chartable.de/unicode-utf8-table.pl?utf8=0x       ¿
    • Using vim editor search for the special character in file and delete it manually.
  • After copying the file, do the required process in the table
    • --delete FROM table_name WHERE length(phone_no)!=10;
    • --delete FROM table_name WHERE length(phone_no)=0;
    • --delete FROM table_name WHERE phone_no is null
    • --delete FROM table_name WHERE length(subscriber_name)>95;
    • --delete FROM table_name WHERE length(address)>295;
  • Alter the column datatype to required one
    • ALTER TABLE table_name ALTER COLUMN phone_no TYPE bigint USING (phone_no::bigint);  //Column name is phone_no
  • Fill unfilled column in database table with value from another column, delemited value
    • update table_name set pincode = ( SELECT split_part(address, '!', 3));
  • Update Column by replacing occurance of a string
    • UPDATE SET = REPLACE ( , 'string1', 'string2' );

Steps for take postgres table dump and import to another Postgres database

  • Login as root user
    • sudo su
  • Login as postgres user
    • su - postgres
  • Take dump of the table to file named output.sql
    • pg_dump -p 5432 --column-inserts --data-only --table=table_name database_name > output.sql
  • Copy output.sql to postgres/data folder of the machine where it is to be ported.
  • Create table at destination with same table name and properties
  • Go inside the /postgres/data folder and give this command
    • psql -p 5444 -d database_name -f output.sql  
      • -p is for port number
      • -d for database name