nohup egrep -rnw '=\s112' --include=*.java ./ 2>&1 | tee ~/112-audit-nick.txt
Category: Uncategorized
-
Recursively GREP for specific content
-
Exponential backoff
In a variety of computer networks, binary exponential backoff or truncated binary exponential backoff refers to an algorithm used to space out repeated retransmissions of the same block of data, often as part of network congestion avoidance.
Here is an implementation in Java
/** * Calculate Exponential backoff * * @param attempt * number that we are checking * @param maxDelayInSeconds * Max amount of time to wait * @param multiplier * How much of backoff to perform * @return */ public static long backoff(final int attempt, final long maxDelayInSeconds, final double multiplier) { final double delayInSec = (Math.pow(2.0, attempt) - 1.0) * .5; return Math.round(Math.min(delayInSec * multiplier, maxDelayInSeconds)); }Example
Here we have exponential backoff defined with three different parameters for the
muliplier and 120 seconds as the max time.System.out.println(String.format("Attempt\t\t 1\t4\t8\n")); for (int i = 0; i < 10; i++) { final long b1 = backoff(i, 120, 1); final long b2 = backoff(i, 120, 4); final long b3 = backoff(i, 120, 8); System.out.println(String.format("%d\t\t %d\t%d\t%d", i, b1, b2, b3)); }1 4 8 0 0 0 0 1 1 2 4 2 2 6 12 3 4 14 28 4 8 30 60 5 16 62 120 6 32 120 120 7 64 120 120 8 120 120 120 9 120 120 120
From the results above we can see that changing the multiplier can have significant implications. The larger the
multiplierthe faster we will be approaching ourmaxDelayand we will have longer paused between each attempt.In next post, we will create a Data Retrieval Service that will utilize Exponential backoff.
-
PhantomSQL 2.0
Since the beginning of this project, I have learned many lessons.
First of all the syntax of the language was to closely related to SQL, as this was the language that I wanted to mimic and fit into this framework. Main reason for that was that I wanted something familiar to the end user, I was hoping for easier adoption rate. Secondly, the lack of Web 2.0+ futures. They were not thought out in the beginning and shoehorning them in just made no sense and even more complicated the language.
Example : Grab data from woot (v1)
foreach @srcUrl : [http://www.woot.com/, http://home.woot.com/, http://shirt.woot.com/, http://kids.woot.com/] begin set @doc = select xpaths("//*[@class='amount']/text()"), xpaths("//h2[@class='fn']/text()"), xpaths("//*[@class='lightBox' and @rel='sale']/img/@src"), xpaths("//*[@class='wootOffProgressBarValue']/@style") from @srcUrl set @images = @document.xpaths("//*[@class='lightBox' and @rel='sale']/@href") set @first = @doc[0] set @price = @first[0] set @title = @first[1] set @image = @first[2] set @wootoff = @first[3].split(":") set @wootoff = @wootoff[1].replace("%", "") insert products (@srcUrl, @title, @price, @wootoff, @image, @images) endCouple problems that this introduced, first of all, we were constrained by the original language structure. Let’s take simple snippet
set @price = @first[0]
It might not look like much but this type of declaration is very declarative and it could easily be removed from the syntax.
Another drawback was the lack of extensibility. Many of the basic functions have been hardcoded into the parser, thus adding any new language extension was dependent on recreating the AST. Only later and ‘dynamic’ methods have been added but again they were an afterthought and seem more like a hack.
There was no way to reuse existing libraries, basically, this means I had to reimplement all the features that someone already did. This only became apparent to me when I needed to run a md5 checksum on one of the downloaded files.
Another issue for me was that there was no good support for collections (List, Set, Dictionary). There was the basic support for ‘list’ and then later ‘dictionary ‘ but again it was not thought out and missed many features.
As we can see there were quite a few design issues that were not thought out in the initial version. Many of this thing have been learned though running the system on real word problems. As long as we are able to take this and adapt I think we will be in good shape.
Roadmap
- Reworking language syntax
- Based on modern languages / frameworks (EcmaScript / Python / Rubby / SQL/ Node.js)
- Web 2.0 (Support for extracting data from dynamic websites)
- Distributed Processing(Will run on a distributed clustering framework)
- Package Manager (PhantomSQL Package Manager(PPM))
- Reworking language syntax
-
Random number between two values
This is a small utility class that allows us to obtain a random number between two values that are uniformly distributed in the range of ‘low’ to ‘high’. This works for floats, doubles and integers.
The inner working of this class are straight forward, our
uniform(int, int)method uses thenextInt(int)method of Random class which already allows us to pass the upper bound. Float and Double work by obtaining a value in range [0.0, 1.0] and then scaling it accordingly between our ‘low’ and ‘high’As this is meant for use in multithreaded environment I am using
java.util.concurrent.ThreadLocalRandomrather thanjava.util.Randomfor performance reasons.import java.util.concurrent.ThreadLocalRandom; import java.util.concurrent.ThreadLocalRandom; public class RandomUtil { public static int uniform(final int low, final int high) { final ThreadLocalRandom rand = ThreadLocalRandom.current(); return rand.nextInt(high - low) + low; } public static float uniform(final float low, final float high) { final ThreadLocalRandom rand = ThreadLocalRandom.current(); return rand.nextFloat() * (high - low) + low; } public static double uniform(final double low, final double high) { final ThreadLocalRandom rand = ThreadLocalRandom.current(); return rand.nextDouble() * (high - low) + low; } public static double nextDouble() { final ThreadLocalRandom rand = ThreadLocalRandom.current(); return rand.nextDouble(); } public static boolean nextBoolean() { final ThreadLocalRandom rand = ThreadLocalRandom.current(); return rand.nextBoolean(); } } -
Procedural lightning effect Unity
DRAFT
This is a basic tutorial on how to create procedural lightning effect in Unity, this is my first attempt at using Unity so if you think there are bugs,issues or better ways of doing things let me know.
I am using a two step process
1) Generator
2) RendererI like to have them separated for couple different reasons but mainly to allow me to render them differently and create different lighting like effects.
Generator is responsible for generating segments and renderer is responsible for rendering segments to the screen.
My original version used a LineRenderer but I decided to go with a Mesh/MeshFilter for rendering as that gives me more control. Each segment creates a new quad that is added to our mesh.
After 1 generation
Results after first pass.
Current status
Implementation
Segment class
public class Segment { public Vector2 start; public Vector2 end; public int generation; public Segment(Vector2 start, Vector2 end) : this(start, end, 0) { } public Segment(Vector3 start, Vector2 end, int generation) { this.start = start; this.end = end; this.generation = generation; } } -
Configuring Java JDK on Ubuntu
This is an easy way to configure java on a linux box, all this information is available online.
First we need to obtain the build.
sudo wget http://192.168.201.47:8000/jdk-7u75-linux-x64.gz
Extract from tar
sudo tar xzvf jdk-7u75-linux-x64.gz
Create symbolic link so we can later update the version
sudo ln -s /opt/jdk1.7.0_75/ /opt/java
we edit the /etc/profile and add following two lines
export JAVA_HOME=/opt/java export PATH=$JAVA_HOME/bin:$PATH
finally we ‘source’ the file
source /etc/profile
At this point you should be ready to go, we can verify this by executing
java -version
-
Grepping for multiple strings in a file
We will use egrep which accepts a regular expression to grep for multiple strings.
tail -f localhost_access_log.txt | egrep "\" 404|\" 500"
Here our example looks at logs to see if we got 404 or 500 request.
"GET /favicon.ico HTTP/1.1" 404 973 "GET /login.html HTTP/1.1" 500 1230 "GET /favicon.ico HTTP/1.1" 404 973
-
Apache 408 Connection timedout
nohup tail -f access.log | grep ‘408’ –line-buffered | awk ‘{split($0,a,” “); print a[1]; fflush()}’ | tee -a bad-408.txt
-
Tokenizing/splitting string in c++
This method uses strtok to tokeninze our string given a specific delimeter, results of that are put into supplied vector. There are few other ways we can do this but this one is straight forward.
#include
#include #include #include #include #include #include -
#include
- apple
- organge
- cherry
using namespace std; void split(vector & out, const string& in, const string& delim) { char* lc = (char*) malloc(in.size()); strcpy(lc, in.c_str()); strtok(lc, delim.c_str()); while (lc) { string s = lc; out.push_back(s); lc = strtok(NULL, delim.c_str()); } free(lc); } int main(int argc, char* args[]) { string str = "apple,organge,cherry"; vector o1; split(o1, str, ","); for (int i = 0; i < o1.size(); ++i) { cout << "token = " << o1[i] < Results
Supplied string : apple,organge,cherry
Delemeter : ","
Output -
Back from the dead
So I been offline for a while, did not update my blog or do anything else. But I am back now. In up coming days I will post what have happen since begging of the year.






