Posted on : By : TCTJr Posted in : Direct Thoughts|Out In The Fringe|Random_Thoughts

A D A P T I B I L I T Y !

Well low and below I was having some tea this morning and going through my thousands of RSS feeds and one of the survival method blogs wrote about the following subject matter:

Why Do People Fail To Adapt?

1. Not seeing what is happening. Sometimes folks just aren’t informed on what is going on. Hard to make logical conclusions and adapt to them if you don’t have the information.

2. Refusal to accept the new reality. Folks will just stick their heads in the sand and hope the problem goes away. Often this happens with a family who can’t afford to maintain their current lifestyle. They will try to borrow and juggle bills and try to hold onto the greased string as long as they possibly can. Like tearing off a band aid going slower makes it worse. Better to have a couple really rough months than a couple pretty rough years.  Also there are those starving farmers in Africa. Ya know the ones you always see on the aids commercials? The ones who keep planting pathetic little gardens on their 1/2 of an acre farm in the Sudan where 9/10 years there is a drought that kills all the crops.

3. They limit their options. They refuse to try the kind of food that is available and starve instead. They refuse to move to find work. They hope old jobs will come back instead of focusing on retraining and finding new employment.

4. They give up – Adapting, at least in the context we are talking about, is usually unpleasant. Like any other human situation, there is a huge psychological factor. When something bad happens, I am not saying you shouldn’t be sad. Eat a carton of ice cream, or drink a half bottle of booze or whatever it is you do. Go to sleep, wake up and get the heck on with your life. It will eventually get better.

Wow.  Say it aint so?!   Lets do an item-by-item analysis here.

Item #1 Not Seeing what is happening.

Being aware and not being stupid.  Ok this is almost a circular argument as solipsism is seeping in here.  Really?  Please make logical and rational decisions. Pretty obvious and appears to be a given if your building software.  In many cases not being informed is another way of saying you do not have the background to process the information.

Item #2 Refusal to accept the new reality.

This is a big one.  In many cases people cannot cope with vast changes in technology or any type of culture.  I mentioned homeostasis in my last blog.  Resistant to change.    It is a conundrum of sorts.  They want better yet they do not want to deal with a new reality or they have the inability to understand a new reality.  This is occurring in the world of database technology.  (thought I would throw in some software terms to make it tech based).  It is hard when someone busts your whole reality of what you thought was possible.  We no longer need huge co-locations to run companies.  EC2 changed what is possible.  People are still fighting it.  Machines that learn?  OMG they will take jobs away!  It reminds me of the 1950s robot scare (not that I was around then I just read about it…)

Item #3 They Limit their options

Doesn’t this appear counterintuitive when you take it out of the context of life and death or not creating competitive software?  Options! Choices! Menus! Features!  Heck Sonic Burger makes a killing on menu features and options.  So does Microsoft Office – something like 1739 features?

Item #4 They give up adapting.

If you just quit, you get what you deserve. This doesn’t mean keep doing the same thing over and over and over and over and expecting a different result – that is crazy.  If you just quit and literally give up you deserve what you get.  Go Home.  Walk on home boy.

There is also a flip side to adaptation and that is knowing when to hold the line and not change to the latest bright shiny object or algorithm.  This is also adaptation albeit using experience.  Being Patient is an adaptive.  Mr Alligatorinae pictured above is a great animal at being patient.  Be Patient – Adapt – Emerge.

Until Then

Go Big Or Go Home.

Original site can be accessed here:

WHY DO PEOPLE FAIL TO ADAPT?

Posted on : By : TCTJr Posted in : Direct Thoughts|General Discussion

Social Darwinism and Software Development

I was having a discussion the other day with a couple of very smart humans.  The discussion trended towards the aspect of performance which naturally led me to start joking about  meetings called “Nurturing Others In The Work Environment” and “I’m Ok – Your OK”.  A very bright software engineer,  on my current team, commented the software speaks for itself, which got me around to the current installment.  The software does speak for itself.  In fact, elite software truly speaks for itself.  Further software development is the ultimate survival of the fittest from an evolutionary standpoint.  For those not familiar with this subject matter I cannot at this juncture give you cliff notes.  Thus we are not nurturing others in the blogosphere.   Mr Spencer who was a pretty smart feller was attributed to coining the phrase “Survival Of The Fittest”.   Stanford is kind enough to give you a great rundown of the subject matter here.  Concerning the taboo terminology of Social Darwinism, we can locate the Wikipedia definition as thus.  Richard Hofstadter devoted an entire chapter of Social Darwinism in American Thought (1955) to Spencer, arguing that Spencer’s unfortunate vogue in late nineteenth-century America inspired Andrew Carnegie and William Graham Sumner’s visions of unbridled and unrepentant capitalism. For Hofstadter, Spencer was an “ultra-conservative” for whom the poor were so much unfit detritus. If you have not read Hofstadter’s book, you should. Once again it is outside the scope of this blog to review but it is one of the few tomes on the “taboo” subject of Social Darwinism.  Also if you have a Facebook account there is a Social Darwinism fan page.

It is my belief that the world of Spencer and Hofstadster via the foundation of Darwin, can be seen in the world of software development.  If you have ever worked on software that touched multiple millions of deployments to billions of feeds you know the importance of quality.  You also know the importance of performance and accuracy.  There is no arguing what happens when someone “breaks the build”.  It is NOT OK – YOU ARE NOT OK.  In fact you are probably going to get your ass called at 2:30 AM after you pulled a 3 day straight coding sprint to get up and fix it or else.  Does this sound like nurturing others in the work environment?  Hell No!  You are costing people money!.   It is a binary process.  Its either on or its off and it better be ON!  All systems go.  I am not saying that we should seek complete perfection, elegance and bug free coding that is impossible if your going to make bank through software creation.  He who ships first will win – if the quality is there.  Yet if your grinding and someone is lagging they are left severely behind the pack and if you watch the discovery channel you know what happens when there are laggers in nature.  It is not pretty.

This is a view from the inside.  From the outside most only see the frisbees and imported beer and sitting there with Pantera playing on the headphones whilst sitting in front of a computing device.  Easy huh?  Think again.  Real coders, data scientist, architects <<insert title de jour here>>  literally have ice running in their veins when it comes to creating software  – usually for money.  As I wrote in other installments, I discussed extreme capitalism and mercenaries.  Good, Better, Best.  It is especially exacerbated when the best of the best make it look easy.  The same for sports.  Kelly Slater, Tiger Woods, Michael Jordan these people make it look so easy.  Aplomb comes to mind.  I am fond of saying, ” To do a dangerous thing with style is the mark of a professional”.

Historically the software industry was a juxtaposition of the peace love and happiness stuff which is in direct conflict with what actually occurs within the world of software. Another interesting aspect to the software world that one must always be A D A P T I N G.  Homeostasis is a death knell to software company.   Microsoft missed the internet but sat down and decided to give away Internet Explorer.  Apple was tanking and created colorful computers.  Companies that continually evolve and change adapt to technology trends and market pressures – win.  Antiquated static slow moving companies are akin to the diseased of a pack and will be taken down.

Dont let you or your company be the one that is taken down.  Evolve. Change. Grow.  Seek Power through great software.  Innovation and Great Code speak volumes.  Amplify those qualities.

Until Then,

Go Big Or Go Home!

Posted on : By : TCTJr Posted in : Direct Thoughts

In Memoriam – Steven Swenson March 13, 1967 – September 11, 2005

Today this blog is not about technology or ideas.  Its a Memoriam.

Most will consider 9/11  as a remembrance of an atrocity that occurred within our nation.  Some cried.  Some died. Some cheered.  Personally, I think 9/11 as it is commonly known within the vernacular and as “The Event”, was a pathetic display of religion and government gone awry.

That said over the years since September 11, 2005 –  I remember it as the day my brother at arms died.

His name was Steven Swenson (a.k.a The Sven).

If you are familiar with OS9, OSX or Windows XP then you are familiar with this man’s works of art.  Steven was an incredible software engineer.  His design ethos as well as his ability to write solid code and adapt in the moment were world class. This year, in homage, I am going to discuss some little known facts and would be fiction depending on whom you talk to concerning some experiences. This is timely, as of late many people have been asking me about how it was to work at Apple, Microsoft, and The Valley in general.  First, I will set the stage with some “Sven-isms” that he used to say or yell depending on the situation:

“Mediocrity Is Not An Option!”

“That is fucking stupid”

( Usually in response to someone’s question if anyone had any comments or feedback  – complete with laughter.)

So with the “level set” a couple of vignettes:

Story #1

We are standing around at “The Evil Empire” and I asked Steven what he had going on with respect to a code base that we had just finished architecting.  He said he had to check in some last minute changes and he was finished.  Then his manager walked in and said that he has a bug in the code.  The following discussion took place:

Sven: “OK, I’ll take a look at it now.”  He told me to wait a minute.

Sven: “Oh boy I am a dumbfuck, OK its fixed.”  (insert technical reason here)

Sven looking at his manager: “Ok its fixed now, it looks like that was the last bug in triage before RTM.”

Manager: “Your six weeks early with the (insert current code base here).  What are you going to do now?”

Sven: “I am going to the gym with TCT.”

Manager: “Then what?”

Sven: “Stay out of here as long as I can.”

Beautiful really.  Just beautiful.

Story #2

We are somewhere in Cupertino, CA hanging out at a certain Loop.  We were working on an new operating system circa 1999.  There was a discussion over how we were going to implement some real time Application Programmer Interfaces.  There appeared to be a disconnect between the two technical lead / managers, the Sven being one of the two at odds.  So we were sitting there arguing.  He gets up and says:

“I am going to beat the shit out of you.”

I get in between them and he starts at the other lead manager.

I grab him and throw him outside the building.

He says, ” I am going to get a gun and kill that motherfucker.”

I punch him and he keeps yelling.

I hold him down and tell him I am going to knock him out if if he doesnt stop.

He said, “Ok but that guy is leading us down a path that will fail.”

I only can hope to find people who care about Quality as much as he did.

Story #3

One year for Xmas I gave my team a shirt that said, “100% Microsoft Free – FuckBillGates.com”

I gave these shirts to them before coming back from lunch at the Evil Empire.

I knew exactly what was going to happen. I dropped The Sven off somewhere around say building 31 and he proceeded to put the shirt on.  He said he had to give design meetings the rest of the day.  I responded that he might not want to do that.  He said “Fuck them if they can’t take a joke.” I called later and inquired about the response.  He said no one said anything that they were a bunch of pathetic followers.

I suppose if the shoe fits.

Think you have created some cool stuff?  Think again.  In that same vein this is how much Steven or ( The Sven ) hated computers:

PC Destruction Video

This was one of his favorite songs:

Pantera: Becoming

Here are the Lyrics to Sing Along with…

A long time ago I never knew myself. Then the memory
Of shame birthed its gift.
No more. The small one, the weak one, the frightened one.
Running from beatings, deflating. I’m becoming more
Than a man. More than you ever were. Driven and burning
To rise beyond Jesus.
I’m born again with snakes eyes
Becoming Godsize
I found my life was slipping through my hands. Perhaps
Through death my life won’t be so bad.
I can see you, can fuck you, inside of you. Staring through
Your eyes. Belittle your friends to serve me, to suck me,
To realize my saving grasp. I of suicide. I the unlord.
I’m born again with snakes eyes
Becoming Godsize

In retrospect, it might have been prescient. He probably would tell me to shut the fuck up.  In addition to being a great computer scientist he was also a great waterman, knew martial arts and was in phenomenal shape.  He was much better than most.  He was elite and a Nihilist at the same time.  He believed Schrödinger’s cat was dead and eviscerated, so to speak.  He thought most were stupid and most were mediocre humans.  There is also this misconception that people who make technology are dorks.  Does this guy sound like a dork?  Nowadays I see so many people wanting to be spoon fed.  I can assure you that if the Sven were alive today he would feed you a bootjack or provide you with an island, a gun with one bullet and a case of rum.  I’ll close with some words from a song by Blood Axis called Comrade:

To every sailor the gods have given a comrade
While one sleeps the other keeps watch on the bridge
When one doubts the other gives him his faith
When one falls the other discovers the oasis of ice for both of them

 

When my comrade loses faith I laugh confidently
When my comrade sleeps I keep watch for him
When my comrade falls I fight on for the both of us
Because to every sailor the gods have given a comrade
Hey!

If you loose heart, I’ll laugh joyfully
And if you sleep, I’ll keep watch for you
And if you fall, I’ll fight on for the both of us
Because to every warrior the gods have given a comrade
Hey!

If you fall I will discover the oasis of ice for both of us
Hey!

Hey!

I truly miss my comrade.

As we used to say “One day as a lion instead of a life as a sheep.”

Go Big Or Go Home!

ALL HAIL STEVEN SWENSON!

Posted on : By : TCTJr Posted in : General Discussion|Human-Performance|Out In The Fringe

Mercenaries For Hire

Blackwater Mercs Likely to Stay in Iraq, Despite Gov't Ban | WIRED

In my last post I discussed talent.  Most companies buy companies either for a defensive move or for talent.  Few and far between are the exemplar purchases for intellectual property.  Although it is mandatory you have those ideas at least patent pending with some form of provisional application – yet i digress.

It has become very apparent to me we are heading into a talent shortage.  Specifically in the areas of Data Science.  It reminds me of the glory days of signal processing in the early to mid 90’s.  We couldnt find people who knew how to code a (fast) Fast Fourier Transform let alone think about distributing that processing.  I see Data Science related careers in that same situation.  I also see the physical and soft sciences merging with the two as it pertains to gaining knowledge from the data.  Need to find a lot of people and do not want to outsource it?  Do not have the cash to make a company acquisition?  Call in a team.

The above picture is of the good folks at Blackwater.  Most people or companies do not like mercenaries.  Why? Mercenaries are usually not very fun.  Most people enjoy pithy campy fun.  Mercenary teams are not the type of consultants you find in Office Space.  On the contrary, these guys and gals know what they are doing – get in – get out and move on.  They are expensive but will usually overhaul any situation.  They have a huge amount of experience and are very calm under chaotic conditions.  I am reminded that those who do a dangerous thing with style is a professional.  I see a day when teams of Mad Max Mercenaries are plying their trade on Craig’s List or eBay.  eBay is probably a better venue as then they can have a cover LLC and incorporate subdivisions as needed based upon specialty.  Sound crazy?  Maybe.  Sounds like a possible Idea2Bank to me.

Until Then,

Go Big Or Go Home!

Posted on : By : TCTJr Posted in : Uncategorized

Google Buys Semantic MetaWeb

This is great news on many fronts. First its about the data – in fact – the metadata associated with the company. Its is also about the almanac nature of the data. MetaWeb was also Resource Description Framework (RDF) based which bodes well for the future of the Semantic Web. Third they were a startup that was purchased. Empowerment. Entrepreneurship. Ideas-2-Bank.

here is the link:

Google Acquires Metaweb To Make Search Smarter

Until Then,

Go Big Or Go Home!

@jaxsoncreole

Posted on : By : TCTJr Posted in : Direct Thoughts|General Discussion|General Technology

Revisiting the T’s with Talent

I usually don’t track “the news” or “chatter” but the recent Facebook acquisition of NextStop is about talent.  Possibly geolocation and hyper-local data but more about who is there and why they are there.  I was having this discussion with several persons the other day.  Companies buy other companies for talent, data aggregation and possibly technology.  The reason I say possibly technology is that most companies do not care about the tech – mostly talent.  I have a term for the companies that move our tech world:  The Folks Inside The Firewall.  The Folks Inside The FireWall (TFITF) already have a long term strategic view of where they want to go 5,10,20,50 years.  Take this to the bank.  In a blog a couple of years go I discussed the Three T’s of a Startup.  I listed talent as number one.  Why?  I prefer quality over quantity.  I prefer the elite performers.  I’d rather have two A game coders than 10 posers.  I located a recent plot of distributed and related coding jobs.  While the plot focuses on Casandra rest assured that the areas of machine learning, data mining, natural language processing and semantics are going to skyrocket.  10 years ago we were worried about connections, storage and compute resources.  This is no longer the case.  We are revisiting this world – yet again – same algorithms – more data.  It reminds me of the hey day of digital signal processing engineers.

As I discussed in my previous blog Three T’s of a Startup many times the companies do not even use the acquired technology.  So please do not take it personally when your great code base is destroyed.  Philosophically Art is meant to be created and destroyed.  The money and vesting schedule will make the destruction worthwhile – trust me.  If you want to go be idealistic go right ahead.  The smart ones wait and vest.  Colloquially called resting and vesting (RnV).  It is a great way to past the time.   That said this brings up the question of is it really about the software now?  Clearly, GOOG does not need a geo-loc technology nor are they going to be a travel agency.  Patents?  Possibly.  In that same blog I discussed how many executives at these companies just say put together a team, write some code, create some provisional patents.  If the stuff halfway does what your roadmap says we will probably buy you.   Look for my next blog on this subject called Mercenaries For Hire – Have compilers – will not travel.

Until Then,

Go Big Or Go Home!

 

Posted on : By : TCTJr Posted in : Direct Thoughts|General Discussion|General Technology

Money It’s A Hit!

 

Why We Do IT

“If you ask me to name the proudest distinction of Americans, I would choose- because it contains all the others- the fact that they were the people who created the phrase “to make money”. No other language or nation had ever used these words before; men had always thought of wealth as a static quantity- to be seized, begged, inherited, shared, looted or obtained as a favor. Americans were the first to understand that wealth has to be created.” ~ Ms. Ayn Rand

I’m not sure about you, but every time I am asked why I do what I do – I always think it sure beats digging ditches.  I mean really folks let’s be honest here.  Would you be doing anything else?  Software is the most scalable profession this world has ever known and in this post-July 4th blog I was thinking – wow America created the industry.  The Epicenter of it all.  I believe that creating software is the epitome of The American Dream.  An Idea, A Computing Device and a Connection.  It is complete freedom.  There are no rules.  The only rules are mandated via the compilers and processors.  The one-to-many multi-cast aspect of creating software and now DataSpaces is completely scalable.  Doctors, Lawyers, and in some cases Stock Brokers are not scalable as many times it is a one to one relationship and is a time allotment with respect to target audience.  If your creation is a hit then there is a non-linearity for scale that cannot be duplicated.  Then comes the after affect.  Money.

Most do not set out to say – “Hey I am going to make some money.”  Nope in the software world you say, “Hey wouldnt if be cool if <insert idea here>.”  I am really getting tired of people trying to soften the aspect of Extreme Capitalism in the software industry.  So for arguments sake which I enjoy let us do a little play acting:

Entrepreneur: I am creating a Data As a Service company for Socnets

Venture Capitalist: So why did you found the Acme DaaS Corporation?

Entrepreneur: To save the White Hooded Wood Owls from extinction.

Venture Capitalist: Wow that is great I can definitely see my 10x return on this $10M Series A investment.  Whoo Hoo!  Lets do it! Woot!

Not realistic huh?

So lets step back here in a minute.  If you want to be a philanthropist and have not inherited a ton of money then creating a really hot software company with a great exit is a way to get there so you can save the White Hooded Wood Owl. (DISCLAIMER: We are not being selective of wood owls please find the brown one here. In fact I do not think there is such thing as a white hooded wood owl).

So what should you say?  Well Mr./Ms. VC I want to sell this and make a ton of…wait….keep guessing…. MONEY!

Don’t posture.  Money Rules the game.

Until Then,

Go Big Or Go Home.

Posted on : By : TCTJr Posted in : Computing_History|Direct Thoughts

Man You Are Lucky

“Wow must be nice, you’re lucky!”

For the successful entrepreneur how many times have you heard this? In the world of “ustas” I used to get really mad about that when someone would say this to me or someone that I knew.  I suppose jealousy and envy cloud the mind.

I have a good friend who is an accomplished martial artist and weapons expert with multiple level black belts in various forms.  He is fond of telling his advanced students training up in the hills in some remote local, “Luck has nothing to do with why you are here.”

There are some who say that having an exit for your startup is complete luck.  Ok fine.  So be it.  We can argue that one later.  What I want to discuss today is what happens post-acquisition with the founders and principles of a startup that has been acquired with the proper frameworks in place.  Lets take a recent example of the high profile IPO of Tesla. The PayPal Mafia is a great example of how to take care of the people that make the company successful.  It is also a great example of how the machine works.  During my last startup here on the right coast I was continually asked why I started the company?  Well lets see: take an idea, create some software, sell it, make money.  Money, Oh yea Money.  Actually what I stated was to create a proper venture capital firm much like a Y-Combinator but with more cash.  A good rule of thumb is to not create a company specifically for the company but create it for the next scenario.  the paypal founders rewarded their people.  Musk only had 11.5% at the time of the 1.5B dollar sale. Yet he got back together with his people and started something else and did it again - Rinse - Cycle-Repeat. It is estimated the collective PayPal Mafia now manages30B in tech funds and companies.   I continually see people who have founded company or are founding companies saying they will not give up the equity.  Your choice.  You must give to get, Yin and Yang, Push Me – Pull You.

PLEASE READ: REWARD THE ELITE PERFORMERS – HEAVILY – THEY WILL NOT LET YOU DOWN!

During the last go around I had a good friend of mine who has amazing experience come up from a semi-retirement from St. Marteen Island.  He called me from the harbor club where he landed and said, “I dont need a salary or contract I just want Equity. We can figure it out later, lets start coding.”

Take that to the bank all day.  I gave him multi-digit percentages.  So for those that are trying to get something off the ground or after years cannot scale because you cannot find the right people.  Give the blood.  Give the equity.  It is not luck.

I’ll leave you with a quote:

” The inferior man’s reasons for hating knowledge are not hard to discern. He hates it because it is complex — because it puts an unbearable burden upon his meager capacity for taking in ideas. Thus his search is always for short cuts. ” ~ H.L. Mencken

Until Then,

Go Big Or Go Home!

@jaxsoncreole

Posted on : By : TCTJr Posted in : Artificial_Intelligence|Direct Thoughts|General Discussion|General Technology

The Way To Do It

The machine in action: ITA hammered it.  Massive 100M Series A raise from the crew at Sequia and Catalyst and then in 2010 Google decides to pick them up for700M after a year long bidding war with the usual suspects. Read it here:

ITA Exits at $700M

That said notice why they were bought:  The Data.  Data provisioning and categorization in a nice neat package.  Fluid data that empowered other sites.  Also note the nature of this deal.  People wonder why The Valley will always be on top.  Most will say because they had a head start.  I say because they will continue to invest in people and ideas.

Until Then,

Go Big Or Go Home.

Posted on : By : TCTJr Posted in : Out In The Fringe

Data as Cash

Is this helpful?

What is your data worth?  Do you know the worth of your data?  I bet if there were a privacy breach I bet you would find out at that moment.  Otherwise, what does the data actually do?  Information storage, for information’s sake, is a copy it is also hoarding.  Information assembled in an intelligent high, affinity fashion is worth a lot these days.  

i will say it is worth more than the software that makes it so.

By definition, what does that mean for the software that accesses that data?  Is it as important as the actions taken on the data?  Is it really about the application itself or the results of the information that the application creates across multiple data stores?

To take this a step further, I believe it is no longer about the data sets in situ but how multiple data sets can be fused together with context, geo-spatial and behavioral information.  It is no longer about the software but what the software does with the data – and lots of it.

Unless you live under a rock, you know that distributed map reduction systems are now the norm for any massive-scale compute data-intensive service.  However, there is a catch here.  Digital lifestyles are one massive data collection opportunity.   These digital breadcrumbs are the stuff of congressional hearings on privacy.  The problem is how we can place a recurring revenue stream on this data.  We inherently know it is the new cash.  Distributed data that describes itself is the DataSpace.  The DataSpace is the vault or vaults which create useful information.  This fluidity of data is a very important concept in the proper functioning of Data as a Service (DaaS).

Knowledge Fusion and Assumptions

Knowledge Fusion is a term that historically draws on many dataspace concepts and techniques developed in other areas such as artificial intelligence (AI), machine discovery, machine learning, search, knowledge representation, semantics, and statistics. It is starkly different from other decision-support technologies in as much as it is not purely retrospective in nature. For example, language-based ad hoc queries and reporting are used to analyze what has happened in the past, answering very specific business questions such as, ‘How many widgets did we sell last week?’ When using these tools, the user will already have a question or hypothesis that requires answering or validation. Knowledge Fusion is very different as it is forward-thinking and aims to predict future events and discover unknown patterns and subsequently build models – these models are then used to support predictive ‘what-if’ analysis, such as, ‘How many widgets are we likely to sell next week?’ based on the context and meaning of the data as it is utilized.  KF also allows this context and meaning to infer further linkages from the initial query.  How is this knowledge fusion processed and accessed?  Why is it needed?  Let’s start with some technical assumptions from the BOOM paper:

1. Distributed systems benefit substantially from a data-centric design style that focuses the programmer’s attention on carefully capturing all the critical state of the system as a family of collections (sets, relations, streams, etc.) Given such a model, the state of the system can be distributed naturally and flexibly across nodes via familiar mechanisms like partitioning and replication.

2. The key behaviors of such systems can be naturally implemented using declarative programming languages that manipulate these collections, abstracting the programmer from both the physical layout of the data and the fine-grained orchestration of data manipulation.

Based on these assumptions, I would like to add some  observations from my experiences:

3. The more data we have the easier the machine learning and data mining algorithms.

4. Clean Data is paramount.  85% of the time spent on creating value out of data is cleansing and creating views and modeling.

Another platform that I believe will greatly see adoption is the folks at Systap, LLC with BigData.  Check it out here. These guys are really looking at the future of fully distributed linked data and massive Resource Description Frameworks.  They are also taking into account concerns such as high availability, transactional processing, B+ trees, and sharding.  i hope to see this trend in the enterprise.

Revenue Models for DataSpaces

A caveat emptor at this juncture:  One of my biggest issues with semantic intelligence, knowledge fusion, knowledge discovery, machine learning and data mining is that people believe it is a MAKE IT SO button! Click here and all your dreams will come true. People believe it will give them business strategy answers or generate the next big thing.  Folks y’all still have to think.

While this is all well and good. What do we base the importance of data upon?

  • Revenue Per Employer (RPE)?
  • Keywords Per Transaction (KPT)?
  • Bulk Rate DataLoad (BRD)?
  • Business Lift Based Query (BLQ)?
  • Affinity Per URI (APU)
  • Insert your favorite monetization acronym here…

Securing The Query

An interesting trend is finally happening in the industry.  Folks are realizing that you must secure those mixed and mashed queries.  At the recent Semantic Technology 2010 Conference, there was a panel discussion on security in the semantic world that specifically dealt with dynamically applying access controls.  These technologies allow people to slice and dice the DataSpaces based on proper access control and the meaning of the data streams.  For a great white paper and overview, please visit this link.

The ability to secure specific person-meaning queries in an of itself will usher in completely new monetization models for data.  I like to call this the Reeses Peanut Butter Cup model.  You have chocolate.  You have peanut butter.  Put them together, and you have something really good.  Remember, data does not have calories just consumes bandwidth, disc space, and compute resources  – depending on usage – your mileage may vary.

Until then,

Go Big Or Go Home!