Adventures of a Software Engineer

Author: greg

  • Grepping for multiple strings in a file

    We will use egrep which accepts a regular expression to grep for multiple strings.

    tail -f localhost_access_log.txt | egrep "\" 404|\" 500"

    Here our example looks at logs to see if we got 404 or 500 request.

     "GET /favicon.ico HTTP/1.1" 404 973
 "GET /login.html  HTTP/1.1" 500 1230
 "GET /favicon.ico HTTP/1.1" 404 973

  • Apache 408 Connection timedout

    nohup tail -f access.log | grep ‘408’ –line-buffered | awk ‘{split($0,a,” “); print a[1]; fflush()}’ | tee -a bad-408.txt

  • Starting jetty via command line an nohup

    Somehow I am getting problems starting Jetty via 

    service jetty start

    We will be using unix command called nohup
    “Nohup is a unix command, used to start another program, in such a way that it does not terminate when the parent process is terminated.”

    I have opted out for using this

    nohup java -jar start.jar -Djetty.port=8085

    while this works it shown an message

    nohup: ignoring input and appending output to `nohup.out'

    to fix that up we need to redirect in put and output to /dev/null

     nohup java -jar start.jar -Djetty.port=8085  /dev/null &

  • Taking heap dump of java process on linux and windows

    Taking a heap dump from console when Java VisualVM and JMX is not available to us.
    We will use following tools

      • jmap
      • jps
      • ps

    Dumping heap requires two steps
    1) Obtaining target process id
    2) Dumping heap for given pid

    First we need to obtain the target process id we would like to dump, here I will show couple ways I like to use.

    ps aux | grep 'java'
-----
userx     29901  6.7 47.0 25418812 3848276 ?    Sl   Mar23  85:42 /opt/java/bin/java -Djava.util.logging.config.

    Here second column indicates our process id (pid)

    Second method that is quite useful to obtain pid for java processes

    uxserx@WS4:/opt/java/bin# ./jps -l
4281 sun.tools.jps.Jps
29901 org.apache.catalina.startup.Bootstrap

    As we see both methods returned us pid of 29901
    Npw to perform the dump we issue our second command

    userx@WS4:/opt/java/bin# ./jmap -dump:format=b,file=/tmp/heapdump-001.hprof 29901
Dumping heap to /tmp/heapdump-001.hprof ...

    At this point we have our heap dump that is ready to be analyzed, for my analysis I use two tools. Eclipse Memory Analyzer (MAT) and Java Visual VM

  • Accessing data of leptonica PIX data

    This is mainly as a reference

    
/**
 * Get Pixel value at given  point
 */
l_uint32 pixAtGet(PIX* pix, int_t x, int_t y)
{
    l_int32 wpl    = pixGetWpl(pix);
    l_uint32* data = pixGetData(pix);
    l_uint32* line = data + y * wpl;
    l_uint32 value = GET_DATA_BYTE(line, x);
    return value;
}

    To set a pixel value we can use this

    /**
 * Set Pixel value at given  point
 */
void pixAtSet(PIX* pix, int_t x, int_t y, byte_t value)
{
	l_int32 wpl     = pixGetWpl(pix);
	l_uint32* data  = pixGetData(pix);
	l_uint32* line  = data + y * wpl;
	SET_DATA_BYTE(line, x, value);
}

  • Tokenizing/splitting string in c++

    This method uses strtok to tokeninze our string given a specific delimeter, results of that are put into supplied vector. There are few other ways we can do this but this one is straight forward.

    #include 
#include 
#include 

#include 
#include 
#include 
#include 
#include 

using namespace std;

void split(vector& out, const string& in, const string& delim)
{
  char* lc = (char*) malloc(in.size());
  strcpy(lc, in.c_str());
  strtok(lc, delim.c_str());
  while (lc)
    {
      string s = lc;
      out.push_back(s);
      lc = strtok(NULL, delim.c_str());
    }
  free(lc);
}

int main(int argc, char* args[])
{
  string str = "apple,organge,cherry";
  vector o1;
  split(o1, str, ",");

  for (int i = 0; i < o1.size(); ++i)
  {
     cout << "token = " << o1[i] <

    Results
    

    Supplied string : apple,organge,cherry
    
Delemeter : ","
    
Output
    

      

    • apple
      • 

    • organge
      • 

    • cherry
      •

  • Back from the dead

    So I been offline for a while, did not update my blog or do anything else. But I am back now. In up coming days I will post what have happen since begging of the year.

  • Calculating partial Hausdorff Distance

    
struct Point
{
	Point(int_t _x, int_t _y) : x(_x), y (_y)
	{

	}

	int_t x;
	int_t y;
};


typedef std::list points_t;

double euclideanDistance(const Point& lhs,const Point& rhs)
{
	 double p1 = std::pow((float)(rhs.x - lhs.x), 2);
	 double p2 =  std::pow((float)(rhs.y - lhs.y), 2);
	 double vd =  std::sqrt(p1 + p2);

	 return vd;
}


double hausdorffPHD(points_t seta, points_t setb)
{
    double maxDistance = 0;

    points_t::iterator afront = seta.begin();
    points_t::iterator aback  = seta.end();

    std::vector ranking;

    for(int_t i=0; afront != aback ; ++afront, ++i)
    {
    	Point* a = *afront;
        double minDistance = std::numeric_limits::max();

        points_t::iterator bfront = setb.begin();
        points_t::iterator bback  = setb.end();

    	for(; bfront != bback ; ++bfront)
	    {
    		Point* b = *bfront;
    		double ed = euclideanDistance(*a, *b);

            if (ed < minDistance)
                minDistance = ed;
	    }

    	ranking.push_back(minDistance);
    }

    std::sort(ranking.begin(), ranking.end());

    double fraction = .7;
    int k = (int) (seta.size() * fraction);
    return ranking[k];
}


double hausdorff(points_t seta, points_t setb)
{
    double habPHD = hausdorffPHD( seta, setb);
    double hbaPHD = hausdorffPHD( setb, seta);
    double distancePHD = std::max(habPHD, hbaPHD);
    printf("hd = %0.4f\t %0.4f\t %0.4f\t \n", distancePHD, habPHD, hbaPHD);
    return distancePHD;
}


int_t main(int_t argc, char_t** args)
{

	points_t seta;
	points_t setb;

	seta.push_back(new Point(1,2));
	seta.push_back(new Point(2, 4));

	setb.push_back(new Point(2, 4));
	setb.push_back(new Point(3, 4));

	double val = hausdorff(seta, setb);
}

  • Compiling Webkit on Windows using Visual Studio 2012

    Just some notes on compiling WebKit on windows with visual studio.

    Issues : 

    C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\include\xrefwrap(431): error C2064: term does not evaluate to a function taking 1 arguments (..\..\win\WebCoreSupport\WebFrameLoaderClient.cpp)
25>          C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\include\functional(239) : see reference to function template instantiation '_Ret std::_Callable_obj<_Ty>::_ApplyX<_Rx,WebCore::PolicyAction>(_V0_t &&)' being compiled
25>          with
25>          [
25>              _Ret=void,
25>              _Ty=int,
25>              _Rx=void,
25>              _V0_t=WebCore::PolicyAction
25>          ]
25>          C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\include\functional(239) : see reference to function template instantiation '_Ret std::_Callable_obj<_Ty>::_ApplyX<_Rx,WebCore::PolicyAction>(_V0_t &&)' being compiled
25>          with
25>          [
25>              _Ret=void,
25>              _Ty=int,
25>              _Rx=void,
25>              _V0_t=WebCore::PolicyAction
25>          ]
25>          C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\include\functional(239) : while compiling class template member function 'void std::_Func_impl<_Callable,_Alloc,_Rx,_V0_t>::_Do_call(_V0_t &&)'
25>          with
25>          [
25>              _Callable=_MyWrapper,
25>              _Alloc=std::allocator>,
25>              _Rx=void,
25>              _V0_t=WebCore::PolicyAction
25>          ]
25>          C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\include\functional(515) : see reference to class template instantiation 'std::_Func_impl<_Callable,_Alloc,_Rx,_V0_t>' being compiled
25>          with
25>          [
25>              _Callable=_MyWrapper,
25>              _Alloc=std::allocator>,
25>              _Rx=void,
25>              _V0_t=WebCore::PolicyAction
25>          ]
25>          C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\include\functional(515) : see reference to function template instantiation 'void std::_Func_class<_Ret,_V0_t>::_Do_alloc<_Myimpl,_Ty,_Alloc>(_Fty &&,_Alloc)' being compiled
25>          with
25>          [
25>              _Ret=void,
25>              _V0_t=WebCore::PolicyAction,
25>              _Ty=int,
25>              _Alloc=std::allocator>,
25>              _Fty=int
25>          ]
25>          C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\include\functional(515) : see reference to function template instantiation 'void std::_Func_class<_Ret,_V0_t>::_Do_alloc<_Myimpl,_Ty,_Alloc>(_Fty &&,_Alloc)' being compiled
25>          with
25>          [
25>              _Ret=void,
25>              _V0_t=WebCore::PolicyAction,
25>              _Ty=int,
25>              _Alloc=std::allocator>,
25>              _Fty=int
25>          ]
25>          C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\include\functional(515) : see reference to function template instantiation 'void std::_Func_class<_Ret,_V0_t>::_Reset_alloc<_Ty,std::allocator>>(_Fty &&,_Alloc)' being compiled
25>          with
25>          [
25>              _Ret=void,
25>              _V0_t=WebCore::PolicyAction,
25>              _Ty=int,
25>              _Fty=int,
25>              _Alloc=std::allocator>
25>          ]
25>          C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\include\functional(515) : see reference to function template instantiation 'void std::_Func_class<_Ret,_V0_t>::_Reset_alloc<_Ty,std::allocator>>(_Fty &&,_Alloc)' being compiled
25>          with
25>          [
25>              _Ret=void,
25>              _V0_t=WebCore::PolicyAction,
25>              _Ty=int,
25>              _Fty=int,
25>              _Alloc=std::allocator>
25>          ]
25>          C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\include\functional(675) : see reference to function template instantiation 'void std::_Func_class<_Ret,_V0_t>::_Reset<_Ty>(_Fty &&)' being compiled
25>          with
25>          [
25>              _Ret=void,
25>              _V0_t=WebCore::PolicyAction,
25>              _Ty=int,
25>              _Fty=int
25>          ]
25>          C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\include\functional(675) : see reference to function template instantiation 'void std::_Func_class<_Ret,_V0_t>::_Reset<_Ty>(_Fty &&)' being compiled
25>          with
25>          [
25>              _Ret=void,
25>              _V0_t=WebCore::PolicyAction,
25>              _Ty=int,
25>              _Fty=int
25>          ]
25>          ..\..\win\WebCoreSupport\WebFrameLoaderClient.cpp(97) : see reference to function template instantiation 'std::function<_Fty>::function(_Fx &&)' being compiled
25>          with
25>          [
25>              _Fty=void (WebCore::PolicyAction),
25>              _Fx=int
25>          ]
25>          ..\..\win\WebCoreSupport\WebFrameLoaderClient.cpp(97) : see reference to function template instantiation 'std::function<_Fty>::function(_Fx &&)' being compiled
25>          with
25>          [
25>              _Fty=void (WebCore::PolicyAction),
25>              _Fx=int
25>          ]

    Patch
    WebFrameLoaderClient.cpp
    Line 97
    – : m_policyFunction(0)
    + : m_policyFunction(nullptr)

    Issue:

    25>WebKitGUID.lib(WebKit_i.obj) : error LNK2005: _IID_IDOMDocumentType already defined in uuid.lib(i_mshtml.obj)
25>WebKitGUID.lib(WebKit_i.obj) : error LNK2005: _IID_IDOMProcessingInstruction already defined in uuid.lib(i_mshtml.obj)
25>WebKitGUID.lib(WebKit_i.obj) : error LNK2005: _IID_IDOMEvent already defined in uuid.lib(i_mshtml.obj)
25>WebKitGUID.lib(WebKit_i.obj) : error LNK2005: _IID_IDOMUIEvent already defined in uuid.lib(i_mshtml.obj)
25>WebKitGUID.lib(WebKit_i.obj) : error LNK2005: _IID_IDOMKeyboardEvent already defined in uuid.lib(i_mshtml.obj)
25>WebKitGUID.lib(WebKit_i.obj) : error LNK2005: _IID_IDOMMouseEvent already defined in uuid.lib(i_mshtml.obj)
25>WebKitGUID.lib(WebKit_i.obj) : error LNK2005: _IID_IDOMMutationEvent already defined in uuid.lib(i_mshtml.obj)
25>WebKitGUID.lib(WebKit_i.obj) : error LNK2005: _IID_IDOMWheelEvent already defined in uuid.lib(i_mshtml.obj)

    Added linker option to WebKitGUID /FORCE:MULTIPLE
    Now we get warning instead of errors;

    13>WebKitGUID.lib(WebKit_i.obj) : warning LNK4006: _IID_IDOMDocumentType already defined in uuid.lib(i_mshtml.obj); second definition ignored
13>WebKitGUID.lib(WebKit_i.obj) : warning LNK4006: _IID_IDOMProcessingInstruction already defined in uuid.lib(i_mshtml.obj); second definition ignored
13>WebKitGUID.lib(WebKit_i.obj) : warning LNK4006: _IID_IDOMEvent already defined in uuid.lib(i_mshtml.obj); second definition ignored
13>WebKitGUID.lib(WebKit_i.obj) : warning LNK4006: _IID_IDOMUIEvent already defined in uuid.lib(i_mshtml.obj); second definition ignored
13>WebKitGUID.lib(WebKit_i.obj) : warning LNK4006: _IID_IDOMKeyboardEvent already defined in uuid.lib(i_mshtml.obj); second definition ignored
13>WebKitGUID.lib(WebKit_i.obj) : warning LNK4006: _IID_IDOMMouseEvent already defined in uuid.lib(i_mshtml.obj); second definition ignored
13>WebKitGUID.lib(WebKit_i.obj) : warning LNK4006: _IID_IDOMMutationEvent already defined in uuid.lib(i_mshtml.obj); second definition ignored
13>WebKitGUID.lib(WebKit_i.obj) : warning LNK4006: _IID_IDOMWheelEvent already defined in uuid.lib(i_mshtml.obj); second definition ignored
13>C:\cygwin\home\gbugaj\WebKit\WebKitBuild\Debug_WinCairo\bin32\WebKit.dll : warning LNK4088: image being generated due to /FORCE option; image may not run

    Building Dependencies

    Cairo

    Issue

    gbugaj@LTRMS7GB /cygdrive/c/cygwin/home/gbugaj/cairo
$ make -f Makefile.win32  CFG=release

make[1]: Entering directory '/cygdrive/c/cygwin/home/gbugaj/cairo/src'

cairo-deflate-stream.c
e:\source\c-libraries\zlib-1.2.3-lib\include\zconf.h(289) : fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory
../build/Makefile.win32.common:55: recipe for target 'release/cairo-deflate-stream.obj' failed
make[1]: *** [release/cairo-deflate-stream.obj] Error 2
make[1]: Leaving directory '/cygdrive/c/cygwin/home/gbugaj/cairo/src'
Makefile.win32:12: recipe for target 'cairo' failed
make: *** [cairo] Error 2

    Fix is to add empty ‘unistd.h’ to zlib include directory

  • Calculate centroid of 2D non crossing polygon

    Calculate centroid of 2D non crossing polygon,
    To accommodate that points are correct using Gift wrapping algorithm(Finding Convex Hull)

    Test case 

    import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;

import java.awt.Point;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

import org.junit.Test;

public class MathUtilTest
{

    @Test
    public void computeCentroidWithHull()
    {
        Point p1 = new Point(1, 1);
        Point p2 = new Point(2, 2);
        Point p3 = new Point(3, 1);
        Point p4 = new Point(1, 0);
        Point p5 = new Point(0, 1);
        Point p6 = new Point(5, 5);

        Point centroid2d = MathUtil.centroid2D(Arrays.asList(p1, p2, p3, p4, p5, p6));
        assertEquals(new Point(2, 1), centroid2d);
    }

    @Test
    public void computeCentroid()
    {
        Point p1 = new Point(1, 1);
        Point p2 = new Point(2, 2);
        Point p3 = new Point(3, 1);

        Point centroid2d = MathUtil.centroid2D(Arrays.asList(p1, p2, p3));
        assertEquals(new Point2D(2, 1), centroid2d);
    }
}

    Implementation

        /**
     * Calculate centroid of 2D non crossing polygon, To accommodate that points
     * are correct using Gift wrapping algorithm(Finding Convex Hull)
     * 
     * @ref http://en.wikipedia.org/wiki/Centroid#Centroid_of_polygon
     * @param vertices
     * @return
     */
    public static Point centroid2D(final List vertices)
    {
        if (vertices == null)
            return new Point(0, 0);

        List hull = null;
        if (vertices.size() < 2)
            hull = new ArrayList(vertices);
        else
            hull = findConvexHull(vertices);

        // Now we can calculate the centroid of polygon using standard mean
        final int len = hull.size();
        final double xy[] = new double[] { 0, 0 };
        for (int i = 0; i < len; ++i)
        {
            final Point p = hull.get(i);
            xy[0] += p.getX();
            xy[1] += p.getY();
        }

        final int x = (int) (xy[0] / len);
        final int y = (int) (xy[1] / len);

        return new Point(x, y);
    }