A face-lift and an update

It’s been right around a year since I posted last in the blog, and I figured it was time for an update, after someone commented on my How to Handle Online Harassment blog post and a code error prevented me from approving and replying.

A couple new things have happened that have kept me out of the how-to-tech world as much as I have been in the past:

First I was married around this time last year. The wedding itself was an exercise in fusing current technology with the 1940s. The wedding ceremony music was timed to the second from start to finish so it could begin on 11/11/11 @ 11:11am. (If only the sound booth guy would have followed instructions, everything would have worked out fine). Everything was instrumental, but varied from Vivaldi’s Four Seasons, to Smashmouth’s Accidentally In Love. I walked out, of course, to the opening credits music from The Matrix: Revolutions. It had to be done.

The reception music was all controlled from my Motorola Atrix Android phone, using some Brookstone Outdoor Wireless Speakers, and a subscription to Grooveshark.com. A slideshow video was created using Animoto.com to add a little more flair and excitement to the otherwise dull “here’s all our photos from our entire life – YOU MUST WATCH THIS BORING THING” video. I hate those – especially the ones that require 3 or 4 songs to finish – mine was 1 song and only spanned our adult lives from just before we met up to the wedding. You can view it below (if you have Flash):

The decorations for the wedding (if they had to be ordered) were all homemade with the materials being ordered online, and mostly from etsy.com. I made our Save the Date notifications and our wedding invitations in Adobe Fireworks, and we created a Facebook page to keep everyone up to date, and a Google Document for handling the RSVPs using the Custom Form interface.

Source: appboy.com

The second major tech-related event in my life was receiving my first iPad (3rd Gen) as my wedding gift to myself when it was released in March. I have had it with me every day, all day, and have been trying to use it as a content production device, but there are just no good apps for anything other than photo editing for my various interests. There are indeed a ton of apps out there for other peoples’ interests, but coding and photography are where mine lie, and I have yet to find a coding app that meets my three feature criteria: SFTP, PHP code highlighting, SVN control – all within the same app.

If you’re aware of any apps that contain all 3 of those features, please let me know in the comments below. I have been searching for nearly 9 months now, and have run across several that meet 2 out of 3 – and as the song goes, “2 out of 3 ain’t bad,” but since there is no file system that spans multiple apps, I need all 3 for it to effectively replace my work laptop.

I did consider starting a “working in the cloud: iPad edition” series, but I don’t have the money that someone like iPad Today has for trying and testing all kids of app recommendations, and there are already fairly comprehensive blogs dedicated to this subject, like AppAdvice.com.

In other tech news, I started backing various projects on Kickstarter.com. So far I have backed 3 projects that I considered useful or unique items.

First to ship, but the 2nd project I backed, was the SmarterStand from smarterstand.com. So far I have not engaged in 100% use of the product because I currently use a DoDoCase for my daily iPad protection. However, I foresee a future blog post here reviewing the product once I do decide to use it daily.

The first product I backed, and likely next to ship, was the popular Pebble Watch. I have been waiting for this since May 2012, and look forward to the day it actually ships. I don’t know if I will get involved in writing any applications for it, but I do plan to test it both with my Android phone and my iPad.

And most recently, a product that hasn’t even met its funding goal yet: Light by Moore’sCloud. If you’re reading this before 12/21/2012, please pledge to back it also…not because of the Mayan Calendar, but because there are 25 days to go (at the time of this post) and they are only 26.3% funded. It is a mobile-controlled (iOS at the moment) lamp that runs LAMP, with an app for controlling the color, timers, and wifi for alerts & additional functionality. It’ll match any part of an image for color choice, as well as cycle through other custom choices. You’ll have to watch the video on their page to get all the details of this beautifully designed product.

And finally in other news, as I mentioned at the start of the post, I was forced to update the blog in general because I had a legit comment on my How to Handle Online Harassment blog post, and I needed to approve and reply to it. In doing so, I removed several plugins, updated several more, and decided to go with the “twenty eleven” blog theme from WordPress this time. It’s simpler, and now my blog works with the WordPress iPad app.

In updating and replying to the aforementioned comment, I noticed that someone cited my article in their own article back in April 2012, along with many other sources. Their article was titled “Social networks: the rise of online harassment“. This marks the first time I’ve been cited in something as a “source” instead of as a link. So, thank you Anna Maria Alba for considering my blog post, and its comments worthy of citation.

Technorati Tags: , , , , , , , , , , ,

Jasper: Just a store platform experience, reworked

Jasper LogoFor the past 3 years, I’ve been working on a store that uses the Yahoo! Store platform. At first, the user interface for the web page design and page object models was easy to grasp (both the prefab versions and the custom ones I had access to), but going through the entire rewrite process was a pain – the biggest of which was the requirement of perfection. If a mistake was made in any field names or data-types it held, the whole object model had to be scrapped and started over. They’ve made minor improvements over the past 3 years to their Merchant Solutions side of things, but they have been extremely minor. One thing that hasn’t changed is the interface to edit pages.

There are two methods of doing the editing, depending on the Yahoo! Store platform that you have. If you’re on the Legacy version, you must go to the page itself, and click edit. This provides you with a series of fields and input boxes for each type of field. No intelligence in the design of what those fields are for or differentiation in how they interact.

If you’re on the Merchant Solutions platform, then you have the option of going directly to the page still, or you can do it in a flat-file pseudo database-like UI which is nothing more than a search-by-field interface, which presents you with a different-looking (but still the same) interface for editing the page.

Over the next few months, I’m going to embark on a web-based interface for importing the legacy store feed (objinfo.xml, which can’t be customized) and see what can be done with the data as far as modifying it, preparing it, cleaning it, and exporting it back to the store. I’m also going to attempt to code it entirely using Kodingen.com. I’ve done a highly customized version of this concept for the company I currently work for (at the time of this post), but none of the code will be reused from that project, and this project will have different, and limited features (in some areas, and expanded in other areas), but for a different data model. In some areas, this application will be more limited (since it’s working with a standardized data format, rather than a customized “catalog.xml” feed from Merchant Solutions. Once this basic version is solid, I may start working on a customized version, or even offering a customizing service to handle the catalog.xml file.

Some things I’d like to do differently with this basic version is:

  • create an install process (a la phpBB)
  • extensive automation, and possibly some AI concepts
  • learn more about object-oriented design and how it can be extended conceptually
  • experiment with some various php and javascript frameworks

There are plenty of other feature sets that I’ve learned and developed in addition to these in the past and plan to use as well. So we’ll see how this goes, and I’ll post some updates here. Depending on what I plan to do with it, I may or may not open-source it. If I do, I’ll host it on Google Code.

Technorati Tags: , , , , , , , , , , ,

How to Connect to SQL through Windows Authenticated ODBC in PHP

For about the last year, I’ve been creating a CMS (content management system), for the automatic management and maintenance of my company’s eCommerce site, on the Yahoo! Store platform. The software imports the entire store automatically, runs a series of cleanup processes with about a dozen different criteria, saves the changes it has made, exports the modified pages, and makes them available to download from Jada’s interface. This automation cuts the need for about 3-4 people doing a weeks’ worth of work, and does it all automatically in 10-30 minutes. The one thing it doesn’t do, is the one cleanup process that takes the longest, and requires the most human thought: comparing every product’s available options on the site, and checking them against actual inventory in the order management software.

Until now.

Our current order management software runs as a MS Access front-end to a MSSQL 2005 server backend through and ODBC DSN connection. This connection has been limited to MS Access and MS Excel application/queries, and thus, was the limiting factor to writing this most-complex cleanup process into Jada. The most difficult part in my development was finding an understandable article describing how to make an ODBC call, and then actually get the data back, in the same simple manner that one makes a MySQL query. The real issue has been once the connection is closed, the result cannot be accessed. I had to find that out the hardway, via Microsoft’s convoluted documentation on using ODBC.

Here’s the code I’ve used to make the ODBC connection in PHP (unfortunately my blog’s template can’t handle actual code right now):

Code Breakdown

We’ll start creating a function that makes an ODBC connection, passes it an SQL query, then parses the data into a table/array and return the array.

Function call

function odbcQuery($sql, $attempt="") {

When we call the function, we’ll provide the SQL Query we want executed, and an optional description of what we’re trying to do with the query. In this way, if it errors, an semantic error will be displayed along with the technical one to help locate the code easily.

Database Connection

// Establish an odbc connection with the database
$link = odbc_connect("My_DSN_Name", "", "");

When running odbc_connect() it takes 3 parameters:

  • the DSN; Server,Port; or Server/SQLInstance
  • username
  • password

When connecting using Windows Authentication & a DSN (as this example is about), there are some caveats and things to remember:

  1. On the web server, the User running the Web Service process needs to be a User with permission to access the SQL Server.
    1. In my case, the user running the web server is SYSTEM, and so the user trying to access the SQL server is “DOMAIN\COMPUTERNAME$“.
    2. There is no password for a SYSTEM account, and so on the SQL Server needs to have a user created named “DOMAIN\COMPUTERNAME$“.
    3. Due to some security concerns, I’ve decided to give the account read-only access to the database I want to access. You’ll need to consult your own IT Administrator or security advisor for your security concerns.
  2. In the odbc_connect() statement, you then only need the name of the DSN (which I assume has already been configured on the Web Server you’re using) , followed by two null-quotes: “”.

This creates an active link via ODBC to the SQL Server…supposedly

Database Connection Checking & Error Handling

if (!$link) {
	die('Could not connect: '.odbc_error().': '.odbc_errormsg());
} else {

Next, we check the link . If it just flat-out doesn’t exist, then we kill the program, throw an error message that will read: “Could not connect: <;insert odbc error code>;: <;insert odbc error text>;”. Otherwise, we move on…

Sending the SQL Query & Checking Response

$data = odbc_exec($link, $sql);
if($data === false) {
	echo "ODBC Query: ".$sql."

"; die("ODBC Query failed: ".$attempt."
Error: ".odbc_error()); } else {

Now that we have a valid link to the database, we’ll send a request for data using the odbc_exec() function. This function sends the connection resource ($link) and the SQL Query we want run ($sql). It will return a “ODBC result identifier” or false.

Since a result identifier could, I assume, be the number 0 (zero) I want to ensure that $data is actually false and not just zero. That’s where the triple === comes in. When doing conditional statements, using == will convert the data being compared into a true/false value, where zero or nothing = false and anything else = true. When you use === you test for an actual boolean value, meaning anything including zero = true and false = false.

If the query failed, and resulted in a false result, we’ll display an error message: “ODBC Query: <;insert actual SQL Query>; // ODBC Query failed: <;insert query description>; // Error: <;insert ODBC error code>;”. Otherwise, we’ll move on…

Parsing the Query Results – Column Headers

// Initialization
$row = $fields = $records = $result = array(); 	

// Get the result's column names
$count = odbc_num_fields($data);
for($x=1;$x<=$count;$x++) {
	$fields[] = odbc_field_name($data, $x);
}

We start off by initializing all the variables we're going to use in the next bit of code, to make sure they're empty.

Then we'll run odbc_num_fields() over the $data to get the number of columns we need to iterate through. For columns, the counting starts at 1, so the for-loop starts at 1.

Iterate through each column name and add it to an array, called $fields:

Array (
	[0] => field_name_1
	[1] => field_name_2
	[2] => field_name_3
)

Parsing the Query Results - Records

// Get the result's data: array[record#][column#] = value
$count = odbc_num_rows($data);
for($x=0;$x<$count;$x++) {
	odbc_fetch_into($data, $row, $x);
	array_push($records,$row);
}

Then we run odbc_num_rows() over the $data to get the number of rows we need to iterate through. For rows, the counting starts at 0, so the for-loop starts at 0.

Iterate through each record row and insert it to a temporary array $row using odbc_fetch_into(). Then take $row and put it into an array of records, $records giving you something like this:

Array (
	[0] =>; Array (
		[0] =>; record_1_column_1
		[1] =>; record_1_column_2
		[2] =>; record_1_column_3
	)
	[1] =>; Array (
		[0] =>; record_2_column_1
 		[1] =>; record_2_column_2
 		[2] =>; record_2_column_3
  	)
	[2] =>; Array (
 		[0] =>; record_3_column_1
 		[1] =>; record_3_column_2
 		[2] =>; record_3_column_3
  	)
 )

Making the data useable

Now that we've got two tables/arrays of data - the field/column names, and each record's array of data - it's time to make it usable in a format that we can consistently expect to be returned. There are two ways to do this. We can create an array listind every record as an array with column_name keys and values

// Return data in the format: array[record_id][column_name] = value
foreach($records as $rid =>; $record) {
	foreach($fields as $key =>; $name) {
		$result[$rid][$name] = $record[$key];
	}
}

or we can list every column as an array of record id's as keys and values.

// Return the data in the format: array[column_name][record_id] = value
foreach($fields as $key =>; $name) {
	foreach($records as $record) {
		$result[$name][] = $record[$key];
	}
}

Personally I find the first option to be more consistent with my results when calling a 2-dimensional result from MySQL queries, so it is the one I have gone with in my example at the start of the post, and in this description.

The foreach() statements describe a compilation of a $result array in this manner:

  1. For each item in the $records array, store the record_id as $rid, and the record array as $record.
  2. Then for each item in the $fields array, store its cardinality as $key and it's value/name as $name.
  3. Then compile an array, iterating through each of the fields, storing this $record's associated cardinality $key's value into the $result array's storage for this record's id ($rid) under the appropriate field name.

It's a lot easier to grasp than it sounds. Basically, take array from the Query Results - Records section, and replace the # with the column name in each: [#] =>; record_y_column_x, but store it as a different array, called $results. The resulting array would look something like this:

Array (
	[0] =>; Array (
		[column1] =>; record_1_column_1_data
		[column2] =>; record_1_column_2_data
		[column3] =>; record_1_column_3_data
	)
	[1] =>; Array (
		[column1] =>; record_2_column_1_data
 		[column2] =>; record_2_column_2_data
 		[column3] =>; record_2_column_3_data
  	)
	[2] =>; Array (
 		[column1] =>; record_3_column_1_data
 		[column2] =>; record_3_column_2_data
 		[column3] =>; record_3_column_3_data
  	)
 )

Close the connection, Return the result

		odbc_close($link);
		return $result;
	}
}

Now that we've stored the data we need from the volatile $data variable returned from the SQL Query into $result, we can close the connection to $link using odbc_close(), and then return $result for the program to do with it what it will.

Conclusion

This is just an example code that explains one way of many to extract a variable 1-2 dimension array of data from your SQL Query, using a Windows-Authenticated ODBC DSN connection. There are many other methods to do this, as well as security fixes, data scrubbing, and other modifications that one would probably want to do.

This is the first function I've written in any language to access an SQL Server via ODBC. This is also a function that has worked in tests, but that I have not yet put into production. I encourage you to take this bit of explanation and massage it into something that fits your needs in the code that you're writing, and don't rely on what I've got here as a written-in-stone example of good production-level code. This sample will change many times before I actually implement it.

Happy Hacking!

Technorati Tags: , , , , , , , , , , , ,

WordPress Plugin: Bad Behavior

Source: dayoldcake.com

Since I started using a WordPress blog back in 2005, I’ve always had the Akismet WordPress plugin installed, and it was the sole provider of my spam protection. It has done an awesome job with an at-this-moment 99.843% accuracy rating, and has blocked 21,215 spam comments of which 6,686 of them were just in the last 6 months.

About a week ago, I found an additional spam blocking plugin that has also been very helpful. This one is called Bad Behavior.

In my observations over the last couple months, it appears that Akismet will block a comment that doesn’t seem to have any correlation to the content of the blog post. This would be why you see posts in your Spam queue that contain no links, no really harmful URLs, and just random text or pointless statements in the body of the comment. I’m sure Akismet is much more complicated than that, though, and I would assume there is a backend database of known spamming IPs/Hosts out there that it may also check against. However, the simplest, and likely initial method of detecting spam is via content.

Not with Bad Behavior. Instead of checking the content of the spam, it looks at the stuff you can’t see – the HTTP Headers, IP, User-Agent String, etc. From their own website…

Bad Behavior analyzes the HTTP headers, IP address, and other metadata regarding the request to determine if it is spammy or malicious. This approach has proved, as one user said, “shockingly effective.” After all, spammers write their bots on the cheap, and have little incentive to code very well. If they could code very well, they probably wouldn’t be spammers.

When Bad Behavior looks at a request, it determines if the request matches a profile of known malicious or spammy activity, and falls outside the bounds of a normal human browsing the web. If so, the request is blocked. But a way out is provided for any human beings with unusual configurations or viruses/Trojans on their computer who may be blocked.

Source: How Bad Behavior Works

Here’s an example of some of the content it has blocked from this very blog…

The image above is using a User-Agent string that includes the Windows version “Windows XP”. Anyone who has done their homework, and makes up a User-Agent string knows that Windows XP is actually Windows NT 5.x where X is the Service Pack number applied. Since Windows XP is not a valid User-Agent String (even though they went to so much trouble to include all the other information in the header), it was blocked.

With this image, the plugin saw that the header was missing the “Accept” statement, telling the server receiving the request what types of files it was willing to accept as a response. Most of the attempts to bot-post that I have seen blocked in the past week or so have been this type of error.

According to the Bad Behavior Benefits and Features page, the plugin runs before any of your PHP-based software (yeah, that’s right, it is available for any PHP-coded site, not just WordPress blogs), so your server never has to respond to a bot just “harvesting data and delivering junk.” Instead the bot is given some 400-style error, and never gets a response from your site.

There are more features and settings that I haven’t had a chance to play around with yet, but if I find it necessary, I’ll create an additional post or add them to this one. I recommend this plugin to go alongside any other spam protection you have in place on your form-driven website or blog.

Technorati Tags: , , , ,

Astaro SG “Joining domain failed”

Astaro LogoI just spent the last 2 business days working on trying to get my Astaro SG 120 v7.507 to re-join the domain, after I deleted the “computer” entry from our Windows 2003 SBS AD. After deleting the computer entry, I rebooted the Astaro box and attempted to join it to the domain (with the same flawless effort I experienced during the initial setup). No luck.

I ran across several Astaro User Forum posts telling people the same things over and over about what to check, and none of it was helping.

I’ll try and give a run through of some error message appearances from the Fallback error log, and what you should do to fix them. If you have more, leave them in the comments below, and I’ll add them.

ads_get_dnshostname: No dNSHostName attribute!

You need to make sure you have a Host Name specified in the FQDN style in the Management > System Settings > Hostname tab


ads_keytab_add_entry: unable to determine machine account’s dns name in AD

  • You need to make sure your Astaro box doesn’t show up as a Computer listed in Active Directory anywhere there are computers listed, then you need to make sure there is a DNS Host (A) entry for your hostname, and also you need to reboot your Active Directory and/or DNS server. (For me, they were the same Windows SBS Box, but a reboot fixed this error message.)

»clock skew too great«

  • You need to make sure the Time & Date on both the Astaro box and the Active Directory server are no more than 5 minutes apart.

»pre-authentication failed«

Make sure the Username/Password you're using to join the domain are correct. They should look like the image above.


Below is the fix that finally worked for me, but I never saw anything in the User Forums (or anywhere else, for that matter) related to these error codes and the Astaro box (only to Samba and Linux Servers).

 

libsmb/clientgen.c:cli_receive_smb

rpc_client/cli_pipe.c:rpc_api_pipe

pipe \lsarpc fnum

  • This means that the Intrusion Prevention System is preventing the Active Directory server from approving the Active Directory join. You need to create an Exception like this:

Go to Network Security > Intrusion Prevention > Exceptions and create a new Exception that Skips "Intrusion Prevention" and list the Active Directory Server in the Source Network area.


I’m sure there are probably more items in the Fallback error logs that folks have encountered. Surprisingly, I didn’t even come across this PDF document detailing a lot of them during my searches until I was researching the error codes for this post. Check that document out and see if your answer is in there.

 

The main rebuttals from most of the “suggestions” you’ll find on the User Forums are:

  1. You don’t need to create a Pre-Win2000 Computer in Active Directory for this to work.
  2. You can’t have an existing computer entry in Active Directory for the hostname you’ve given your Astaro box.
  3. You should create a DNS Host (A) entry for the Astaro box, if you run a DNS server outside of the Astaro box
  4. The Time & Date on both the Server and Astaro Box need to be within 5 minutes of each other.
  5. You don’t have to be able to ping the Astaro box from the server for this to work.
  6. You do have to be able to ping the Active Directory server from the Astaro box’s Support > Tools area
  7. Domains ending in .local will work.
  8. You don’t have to use all CAPS when filling in the hostname  & domain name, but it’s suggested by the Astaro people. It won’t make a difference to DNS – CAPS or lowercase both resolve the same.

As I mentioned above, if you have any other error messages or tips or suggestions, etc., related to getting the Astaro box on the domain, please post them in the comments. Too many people have spent too long trying to get things to work simply because the information wasn’t out there when they were searching.

Technorati Tags: , , , , , , , , , , , , , , , , , , ,