Networking stuff for the day


How to find corresponding mac address from given IP address on system?

ARP stands for Address Resolution Protocol, which is used to find the media access control address of a network neighbor for a given IPv4 Address.

On Ubuntu this is how you find mac address
$>ifconfig -a | grep HWaddr

What is a NAT IP address?

Network Address Translation (NAT) is the process where a network device, usually a firewall, assigns a public address to a computer (or group of computers) inside a private network. The main use of NAT is to limit the number of public IP addresses an organization or company must use, for both economy and security purposes.

Quick guide to Websphere MQ


It is the Messaging style of communication the communication takes place between Queue Managers.

Firstly the application program writes to the some name queue then the queue manager determines the location of those queues and handles all of the communication.

MCA ( Message channel agent ) is the Queue Manager program to perform the network communication.

If messages to be delivered to the remote queue the queue manager checks it and put it into the transmission queue to process further.

MQPUT (write) to MQGET (read)

IBM WebSphere reference architecture

http://publib.boulder.ibm.com/infocenter/wmqv7/v7r1/index.jsp

Synchronous

Asynchronous

SOA

Reduce the service to bare business logic

Resource Manager Or Queue manger owns the queue (MCA – Message channel agent)

MQPut

MQGet

Application –> MQ Interface (MQI) –>Queue Manager –> Process Definition –> Queues

Major Calls

  • MQCONN
  • MQCONNX
  • MQDICS
  • MQOPEN
  • MQCLOSE
  • MQPUT – Write
  • MQPUT1
  • MQGET – Read
  • MQSUB
  • MQSUBRQ

Minor Calls

  • MQBEGIN
  • MQCOMIT
  • MQBACK
  • MQINQ
  • MQSET

MQ Messages = Header + Application Data

Header – Message descriptor

WebSphere MQ largest message size is 100 MB

Easily process data in parallel

Conversation

Call and return

Messaging

Program MQI

retrieve messages from queue

Lets now jump in and create the real life example

I will share an example of StockQuote (i.e. stock price)

We require IBM WebSphere MQ Explorer 7.1 software

1)

StockQuotes
1414

****************************************
* Command: “E:\Program Files\IBM\WebSphere MQ\bin\crtmqm” -sa StockQuotes
****************************************
WebSphere MQ queue manager created.
Directory ‘E:\Program Files\IBM\WebSphere MQ\qmgrs\StockQuotes’ created.
The queue manager is associated with installation ‘Installation1’.
Creating or replacing default objects for queue manager ‘StockQuotes’.
Default objects statistics : 74 created. 0 replaced. 0 failed.
Completing setup.
Setup completed.
exitvalue = 0
****************************************
* Command: “E:\Program Files\IBM\WebSphere MQ\bin\strmqm” StockQuotes
****************************************
WebSphere MQ queue manager ‘StockQuotes’ starting.
The queue manager is associated with installation ‘Installation1’.
5 log records accessed on queue manager ‘StockQuotes’ during the log replay phase.
Log replay for queue manager ‘StockQuotes’ complete.
Transaction manager state recovered for queue manager ‘StockQuotes’.
WebSphere MQ queue manager ‘StockQuotes’ started using V7.1.0.0.
exitvalue = 0
****************************************
* Command: “E:\Program Files\IBM\WebSphere MQ\bin\runmqsc” StockQuotes
* Input: DEFINE LISTENER(‘LISTENER.TCP’) TRPTYPE(TCP) PORT(1414) CONTROL(QMGR)
****************************************
5724-H72 (C) Copyright IBM Corp. 1994, 2011. ALL RIGHTS RESERVED.
Starting MQSC for queue manager StockQuotes.
1 : DEFINE LISTENER(‘LISTENER.TCP’) TRPTYPE(TCP) PORT(1414) CONTROL(QMGR)
AMQ8626: WebSphere MQ listener created.
One MQSC command read.
No commands have a syntax error.
All valid MQSC commands were processed.
exitvalue = 0
****************************************
* Command: “E:\Program Files\IBM\WebSphere MQ\bin\runmqsc” StockQuotes
* Input: START LISTENER(‘LISTENER.TCP’)
****************************************
5724-H72 (C) Copyright IBM Corp. 1994, 2011. ALL RIGHTS RESERVED.
Starting MQSC for queue manager StockQuotes.
1 : START LISTENER(‘LISTENER.TCP’)
AMQ8021: Request to start WebSphere MQ listener accepted.
One MQSC command read.
No commands have a syntax error.
All valid MQSC commands were processed.
exitvalue = 0

Putty and Plink command line options


Putty can be used as remote ssh client tool.

However today I learn few tricks on it.

It can be used as command line tool to execute script on remote server.

command prompt > putty.exe -t -ssh -l user -pw passwd -m C:\Users\irfan\Documents\run.txt host 

*Replace user, passwd and host with actuals.

Carate run.txt file in your documents directory and enter few commands in it.

In this case I have C:\Users\irfan\Documents\run.txt with date and  ls commands.

Putty command line options

Above command will open putty application, logged into host based on credentials to execute commands from run.txt file.
But it will not return anything back to command prompt.

So what should be I doing to return the output back to command prompt

command prompt > plink.exe -l user -pw passwd -m C:\Users\irfan\Documents\run.txt host 

*Replace user, passwd and host with actuals.

This command will return the output to your command prompt.

Plink command line options

Enjoy!

The Financial Information eXchange (FIX) Protocol


It is a series of messaging specifications for the electronic communication of trade-related messages.

Example of FIX message (Tag Value pair)

8=FIX.4.1^9=0235^35=D^34=10^43=N^49=VENDOR^50=CUSTOME^56=BROKER^52=19980930-09:25:28^1=XQCCFUND^11=10^21=1^55=EK^48=277461109^22=1^54=1^38=10000^40=2^44=76.750000^59=0^10=165

Fix message divided into 3 main category

  1. Header
    • Fix version
    • sender
    • receiver
    • type of message
  2. Body
    • order type
    • symbol
    • qty
    • price
  3. Footer
    • checksum – used to avoid the transmission error

Let us now understand above example FIX message in detail

(^) caret is a field delimiter

Referring to FIX specification at http://fixprotocol.org/FIXimate3.0/?language=en&version=FIX.4.1

8=FIX.4.1

Tag Description Valid values
8 Identifies beginning of new message and protocol version. ALWAYS FIRST FIELD IN MESSAGE. (Always unencrypted)

9=0235

Tag Description Valid values
9 Message length, in bytes, forward to the CheckSum field. ALWAYS SECOND FIELD IN MESSAGE. (Always unencrypted)

35=D

Tag Description Valid values
35 Defines message type. ALWAYS THIRD FIELD IN MESSAGE. (Always unencrypted)Note: A “U” as the first character in the MsgType field (i.e. U1, U2, etc) indicates that the message format is privately defined between the sender and receiver.
D = Order – Single [NewOrderSingle]

34=10

Tag Description Valid values
34 Integer message sequence number.

43=N

Tag Description Valid values
43 Indicates possible retransmission of message with this sequence number
N = Original transmission [OriginalTransmission]

49=VENDOR

Tag Description Added Depr. Enums from tag Valid values
49 Assigned value used to identify firm sending message. FIX.2.7

 

50=CUSTOME

Tag Field Name Data Type Union Datatype Description Added Depr. Enums from tag Valid values

56=BROKER

Tag Description Valid values
56 Assigned value used to identify receiving firm.

52=19980930-09:25:28

Tag Description Valid values
52 Time of message transmission (always expressed in GMT)

1=XQCCFUND

Tag Description Valid values
1 Account mnemonic as agreed between broker and institution.

11=10

Tag Description Valid values
11 Unique identifier for Order as assigned by institution. Uniqueness must be guaranteed within a single trading day. Firms which electronically submit multi-day orders should consider embedding a date within the ClOrdID field to assure uniqueness across days.

21=1

Tag Description Valid values
21 Instructions for order handling on Broker trading floor
1 = Automated execution order, private, no Broker intervention [AutomatedExecutionNoIntervention]

55=EK

Tag Description Enums from tag Valid values
55 Ticker symbol

48=277461109

Tag Description Enums from tag Valid values
48 CUSIP or other alternate security identifier

22=1

Tag Description Valid values
22 Identifies class of alternative SecurityID
1 = CUSIP [CUSIP]

54=1

Tag Description Valid values
54 Side of order
1 = Buy [Buy]

38=10000

Tag Description Added Valid values
38 Number of shares ordered FIX.2.7

40=2

Tag Description Valid values
40 Order type.
2 = Limit [Limit]

44=76.750000

Tag Description Valid values
44 Price per share

59=0

Tag Description Valid values
59 Specifies how long the order remains in effect. Absence of this field is interpreted as DAY.
0 = Day [Day]

10=165

Tag Description Valid values
10 Three byte, simple checksum (see Appendix B for description). ALWAYS LAST FIELD IN RECORD; i.e. serves, with the trailing <SOH>, as the end-of-record delimiter. Always defined as three characters. (Always unencrypted)

Different message type

Admin Messages

  • Login
  • Logout
  • Heartbeat

Application Message

  • New order
  • Order cancel
  • Execution
  • All business related messages

Do come back to see more updates on this article.

Query Twitter data using Perl


Search Twitter using Perl module Net::Twitter

Pre-requisites

I strongly recommend to read below url before playing around this example

Streaming API Concepts

Net::Twitter – A perl interface to the Twitter API

Streaming API Response Codes

#!/usr/bin/perl -w

use Net::Twitter;
use Scalar::Util 'blessed';

my $nt = Net::Twitter->new(
      traits                 => [qw/OAuth API::REST API::Search/],
      consumer_key           => "PZBzeiUFkzZ0vF4bNY88rQ",
      consumer_secret        => "2VjWny5UTCk4eIde6GBUFdRsCwiOX6jIgQCPmT6VA",
      access_token           => "349126638-MZ0fCAYRQnyarDjsph5sXtOZBF0h6TjbXjxUXHGn",
      access_token_secret    => "r0SyKoqU91uMksfXD8mTPOI9k0h1UkOkb5rH1LVJh8",
      ssl                    => '1',
      decode_html_entities   => '1',
);

eval {
      my $response = $nt->search("India");

      print
      $response->{'results'}->[$_]->{'created_at'}," - ",
      $response->{'results'}->[$_]->{'from_user'}," - ",
      $response->{'results'}->[$_]->{'text'}, "\n\n" foreach (1..10);
};

if ( my $err = $@ ) {
      print "12345";
      die $@ unless blessed $err && $err->isa('Net::Twitter::Error');
}

It will search “India” in latest tweets

Output format

Date TimeUser – Tweet

Mon, 07 May 2012 07:27:37 +0000 - kpark_kunal - @iSaakshi I doubt!india has very concentrated oil importers! Saudi, Iran,kuwait, Nigeria contributing more than 80%(source:PPAC) @Equateall

Mon, 07 May 2012 07:27:37 +0000 - kpark_kunal - @iSaakshi I doubt!india has very concentrated oil importers! Saudi, Iran,kuwait, Nigeria contributing more than 80%(source:PPAC) @Equateall

Mon, 07 May 2012 07:27:36 +0000 - goonergurl20 - @ReiceCookieCook Pls sign this petition & help us #IndianGooners to get ESPN to telecast WBA vs. Arsenal LIVE in India! http://t.co/fq8qcKbQ

Mon, 07 May 2012 07:27:34 +0000 - WildNavigator - @sophontrack :) have it before going to sleep. In India we call this grandmothers medicinal therapies - it works wonders :) plants are gr8

Mon, 07 May 2012 07:27:33 +0000 - NattaliaLopeez - Perra, gata, zorra, wila, lobuki, puta, pendeja, india .i. Eso eres y mas >:O

Mon, 07 May 2012 07:27:33 +0000 - NandiniNitwh02 - Amarnath Yatra 2012 http://t.co/HFXZA28n via @AmarnathYatra1

Mon, 07 May 2012 07:27:32 +0000 - RealRediffCom - AI to debut Dreamliner on domestic routes: Air India says it would be operating its new Boeing-787 Dreamliner ai... http://t.co/FnibB4Kd

Mon, 07 May 2012 07:27:32 +0000 - RealRediffCom - Cut down Iranian oil purchases: Hillary to India: US Secretary of State Hillary Clinton on Monday urged India to... http://t.co/cX3hXbtl

Mon, 07 May 2012 07:27:31 +0000 - korean_fanatics - Yuri sebagai anak pemimpin desa India dalam kehidupan sebelumnya. Tidak heran ia memiliki kulit coklat susu #yulfact

Mon, 07 May 2012 07:27:30 +0000 - CGMANTRA - #CGMantra is the only academy in #India offering training in all domains of #Entertainment & #Media Sector.

Mon, 07 May 2012 07:27:29 +0000 - goonergurl20 - @KeroBlack Pls sign this petition & help us #IndianGooners to get ESPN to telecast WBA vs. Arsenal LIVE in India! http://t.co/fq8qcKbQ

&nbsp;

Have fun!

Upgrade Perl modules using cpan autobundle


It’s become bi weekly task to keep updated your Perl distro.

To achieve it I use following commands

perl -MCPAN -e shell
cpan> autobundle

Above command will generate a snapshot pm file. This file will include list of modules require upgrade.

Example
Snapshot_2012_05_01_01.pm

You can recheck the file and list of module with respective version. You have a facility to remove any of unwanted packages to your distro.

Once done the modification of the file you save it.

Now run below command

cpan> install Bundle::Snapshot_2012_05_01_01

This command will take care of installation of all these module incorporated in Snapshot_2012_05_01_00 file.

Bingo thats all! WeW

Perl6 rakodu on Mac OS X version 10.7.3


  1. download rakudo perl6 src from git
    https://github.com/downloads/rakudo/star/rakudo-star-2012.04.tar.gz

  2. perl Configure.pl –prefix=<DIR> –gen-nqp –gen-parrot

    <DIR> directory where you want to push the binaries of perl6
    It will avoid clash with existing Perl version running in system
    After command executed successfully check prefix dir, it should show following dir’s
    drwxr-xr-x 3 root staff 102 Apr 29 23:06 share
    drwxr-xr-x 3 root staff 102 Apr 29 23:06 include
    drwxr-xr-x 6 root staff 204 Apr 29 23:06 lib
    drwxr-xr-x 3 root staff 102 Apr 29 23:06 src
    drwxr-xr-x 14 root staff 476 Apr 29 23:08 bin

  3. make
  4. make rakudo-testAll tests successful.
    Files=23, Tests=227, 15 wallclock secs ( 0.09 usr  0.06 sys + 11.54 cusr  2.86 csys = 14.55 CPU)
    Result: PASS
  5. make rakudo-spectest
  6. make install

MySQL processes slowing down host server


Site is overloading the server

MySQL InnoDB database with approx 2gb data. Quite a few sleeping MySQL processes that could be the cause.

Before drilling down in code I would like to make few tweaks to mysql db configuration.Reduce the MySQL wait_timeout value to 3 seconds and increasing the number of allowed connections.Apparently, persistent connections do not currently play nice with transactions with InnoDB.

Mysql configuretion file my.cnf needs following parameters to changewait_timeout = 30 #30 secondsIncrease the number of allowed connection.

Here is the script to check if there is a MySQL sleeping process from long kill them and write log


#!/usr/bin/php -q

<?

$link = mysql_connect (“localhost”, “root”, “password”);

$query = “show processlist”;

$result = mysql_query($query) or die (“failed $query”);

while ($row = mysql_fetch_array($result))
{
    if (($row['Time'] >1000 && $row['Command'] == “Sleep”) && ($row['User'] != “Root”))
    {
        $killed = 0;
        mysql_query(“kill “.$row['Id']) or ($killed = 1);
        
        if ($killed = 1)
        {
            echo
            “Killed process number “.$row['Id'].”, with a User of “.$row['User']
            .” in database “.$row['db'].”, which had a time of “.$row['Time']
            .”, and a command of “.$row['Command'].”\r\n\r\n”;
        }
    }
}

@mysql_close($link);

?>