Introducing simpleFM, PHP edition
On Friday the 13th of April, I did a presentation of a complex FileMaker and Flash integration project I built. From previous experience presenting multi-technology integration projects to developers, I have come to realize that what people want is less of the Power Point, less of the gee-whiz demo files, and more of the practical cook book material. So on Thursday before the presentation, I put together a very simple Flash demo which contains all of the details on how to make Flash interact bi-directionally with FMSA, and not much else.
The process got me thinking; it is so simple to understand the well documented FMSA CWP API and build command urls, and it is so simple to use the built-in XML() class in ActionScript 2 to slice and dice the xml FMSA returns, why are all the connectors which exist for interacting with FMSA so relatively complex? I have experience with FX.php, FMandPHP, and FileMakerPHP. For dot net integration with FMSA, there is Wim Decorte's FMdotNET. I also just heard about a brand new, as yet unfinished, open source Flex class for FMSA.
All of these solutions make the problem more complicated by introducing lots of built in features and in some cases introducing a new object class with methods and properties of its own. This just strikes me as unnecessary. Flash, PHP, Flex, dotNet, etc. can readily assemble the simple FMSA command URLs defined in the API, and it goes without saying that they can already do whatever needs to be done once the data is parsed into a structure they understand. So what's up with all the abstraction? Further more, the arrays that many of these connectors give back often need to be massaged anyway, because they generally accept the structure defined in the default FMSA grammar. I think in addition to the impulse to add features and make hammer factory factories, some of these classes are complicated by a backwards compatibility requirements (several support FMS5 and PHP4 for example).
The available XML classes in PHP5, ActionScript2, and ActionScript3, leave little room for improvement in my opinion. I see no advantage to reinventing the wheel, and I see little reason to invent a new class. So to test my theory, over the weekend, I made a single purpose built PHP function called simpleFM which does not suffer from any of these issues. In about 100 lines of code I created a function that includes error handling, and returns a compound query result with an indexed and associative (using recid as a key) version of the resultset data. It is based on SimpleXML functions in PHP5.
It doesn't handle repeating fields (just returns the first rep if it sees one), but that is just a design choice I made. I may decide to change simpleFM to return fields as arrays, but for now I'm thinking that warrants a different function like simpleFMrep(). It also doesn't handle value lists, but why should it? That should be a different function in my opinion. Value lists use a completely different grammar after all. I'll make simpleFMvlist() for that as soon as I need it. simpleFM does handle portals really well. The 2 resultset arrays it returns looks like this:
$query['index'][$index]['fieldname']['portalname']['index'][$p_index]['fieldname']
Or you can access the data by record ID like this:
$query['recid'][$recid]['fieldname']['portalname']['index'][$p_recid]['fieldname']
It also returns some metadata, and the raw results. Here is a complete breakdown of the main query result array:
Now that I have come up with the idea for this simplified function in AS2 for Flash, and done a proof of concept in PHP5, my next trick will be to create an AS3 function that creates a dataProvider object for Flex. I'm more conversant in PHP, so I wanted to work out the concept there first, but I have been noodling with Flex already. The main problem in doing this for Flex is that the results need to be reorganized as XML, in a structure that differs from that created by FMSA. The PHP5 function simpleXML() and the AS2/3 funtion XML() convert XML to an object that can be processed with normal property selectors and array iterators. As far as I can tell, in order for Flex to consume an data, it must be re-factored as XML. In PHP5, we have the asXML() function, so I can easily convert an object back to XML like this:
$object->asXML()
It occurred to me to use a custom xslt on the FMSA, which would return an XML grammar that Flex is able to consume directly, but I have essentially dismissed this idea as being too dependent on access to the datasource. I think the trick for Flex is to handle it on the client side by compiling the parser function into the swf in the same way we already do with Flash. I am confident that it's just a matter of time before I have a similarly simple function in AS3 which returns a Flex dataProvider object from a call to FMSA. Stay tuned, or post here if you beat me to it :)
Update: I emailed a few colleagues and requested feedback. Here's the email I sent :
The process got me thinking; it is so simple to understand the well documented FMSA CWP API and build command urls, and it is so simple to use the built-in XML() class in ActionScript 2 to slice and dice the xml FMSA returns, why are all the connectors which exist for interacting with FMSA so relatively complex? I have experience with FX.php, FMandPHP, and FileMakerPHP. For dot net integration with FMSA, there is Wim Decorte's FMdotNET. I also just heard about a brand new, as yet unfinished, open source Flex class for FMSA.
All of these solutions make the problem more complicated by introducing lots of built in features and in some cases introducing a new object class with methods and properties of its own. This just strikes me as unnecessary. Flash, PHP, Flex, dotNet, etc. can readily assemble the simple FMSA command URLs defined in the API, and it goes without saying that they can already do whatever needs to be done once the data is parsed into a structure they understand. So what's up with all the abstraction? Further more, the arrays that many of these connectors give back often need to be massaged anyway, because they generally accept the structure defined in the default FMSA grammar. I think in addition to the impulse to add features and make hammer factory factories, some of these classes are complicated by a backwards compatibility requirements (several support FMS5 and PHP4 for example).
The available XML classes in PHP5, ActionScript2, and ActionScript3, leave little room for improvement in my opinion. I see no advantage to reinventing the wheel, and I see little reason to invent a new class. So to test my theory, over the weekend, I made a single purpose built PHP function called simpleFM which does not suffer from any of these issues. In about 100 lines of code I created a function that includes error handling, and returns a compound query result with an indexed and associative (using recid as a key) version of the resultset data. It is based on SimpleXML functions in PHP5.
It doesn't handle repeating fields (just returns the first rep if it sees one), but that is just a design choice I made. I may decide to change simpleFM to return fields as arrays, but for now I'm thinking that warrants a different function like simpleFMrep(). It also doesn't handle value lists, but why should it? That should be a different function in my opinion. Value lists use a completely different grammar after all. I'll make simpleFMvlist() for that as soon as I need it. simpleFM does handle portals really well. The 2 resultset arrays it returns looks like this:
$query['index'][$index]['fieldname']['portalname']['index'][$p_index]['fieldname']
Or you can access the data by record ID like this:
$query['recid'][$recid]['fieldname']['portalname']['index'][$p_recid]['fieldname']
It also returns some metadata, and the raw results. Here is a complete breakdown of the main query result array:
$query['url']; //string
$query['error']; //int
$query['errortext']; //string
$query['count']; //int
$query['fetchsize']; //int
$query['index']; //compound array
$query['recid']; //compound array
$query['raw']; //compound object
$query['xml']; //xml
Now that I have come up with the idea for this simplified function in AS2 for Flash, and done a proof of concept in PHP5, my next trick will be to create an AS3 function that creates a dataProvider object for Flex. I'm more conversant in PHP, so I wanted to work out the concept there first, but I have been noodling with Flex already. The main problem in doing this for Flex is that the results need to be reorganized as XML, in a structure that differs from that created by FMSA. The PHP5 function simpleXML() and the AS2/3 funtion XML() convert XML to an object that can be processed with normal property selectors and array iterators. As far as I can tell, in order for Flex to consume an data, it must be re-factored as XML. In PHP5, we have the asXML() function, so I can easily convert an object back to XML like this:
$object->asXML()
It occurred to me to use a custom xslt on the FMSA, which would return an XML grammar that Flex is able to consume directly, but I have essentially dismissed this idea as being too dependent on access to the datasource. I think the trick for Flex is to handle it on the client side by compiling the parser function into the swf in the same way we already do with Flash. I am confident that it's just a matter of time before I have a similarly simple function in AS3 which returns a Flex dataProvider object from a call to FMSA. Stay tuned, or post here if you beat me to it :)
Update: I emailed a few colleagues and requested feedback. Here's the email I sent :
Hi PHP gurus,
Check out this PHP function I set up.
http://www.jsmall.us/downloads/simpleFM_20070413.zip
I posted some thoughts about it on my blog at http://jsmall.us
In a nutshell, the way we connect to FMSA in Flash is so simple and low tech, it got me thinking about why the generally accepted connector classes are as complicated as they are.
So I made a single function that takes care of the main thing I care about, which is parsing out the result. I don't care about all the nifty methods for assembling the command url. IMO, that's easier and more trouble free if you just assemble the command string you need and put in a few variables as needed, and call it a ball game.
Right now, this function returns a pretty verbose result, because it gives an indexed version, an associative version (by recid), a raw dump, and pure xml. The last two are really just in there for development curiosity and possible forensics. But even if they get removed, it still returns two versions of the results for now. I'm thinking of making the indexed version be the default and creating a switch if you want the associative recid version back.
Can you guys check it out and give me some feedback before I offer it to everyone else?
Thanks!
Jeremiah
PS: Next I am going to make the same kind of thing to build a dataProvider object for Flex, so any insight relative to that appreciated.
Comments
I'd suggest that the areas where this needs improvement are:
(1) Don't ever use 'die' in a function like this. Throw an exception or
return null or return an error of some sort. I prefer exceptions, but
that would make it PHP 5 specific.
(2) Find a way to use "POST" rather than a "GET" request. "GET"
requests limit the length of the URL, so you can only put so much data
in the query string. Unfortunately, that probably means you can't use
'simplexml_load_file' but some other mechanism. You can still use the
simplexml library to parse the result.
(3) Allow the user to choose between http and https for the protocol.
(4) Flexibility to make requests and parse responses from other versions
of FileMaker.
It's still an excellent basis for a new library.
=thomas
have an extended function that does (sometimes it's a hassle to use
exceptions, or people just don't like them).
I agree with Thomas' on the POST instead of GET. There's a collection of
comments about using posts here:
http://us.php.net/manual/en/function.fsockopen.php
One more test that would be nice is for mal-formed xml. It's
particularly difficult to determine if the xml returned by the query is
bad or if the connection failed when they are treated as the same type
of error.
A possible way is to retrieve the XML first, then use
simple_xml_loadstring() and if it fails and the string is not empty,
assume bad data, otherwise failed connection.
Something like that.
-tv
Wow, this is nice!
I agree with Thomas and Timmy.. I'd throw exceptions upon error. It is
cleaner, and would also make error trapping easier when writing scripts that
call upon the function.
Jamie
I won't be making any of the suggested changes this week... Too busy!
If any of you get inspired, feel free to make changes. I will post any new versions on the blog.
Thanks!
Jeremiah
It's already PHP5 only, so exceptions are fine and a good idea.
POST would be good too, but I do want to keep it simple, and I worry about ballooning the number of arguments. I guess there is no technical reason to worry about that, just a desire to keep simpleFM simple.
It would be nice to have an https option.
I don't think I care about parsing the older FMS grammar that requires the METADATA->FIELD nodes to understand the names of the RESULTSET->COL nodes instead of just using the 'name' attribute of resultset->field. The argument for not supporting the older grammar would again be to keep it simple, but I could be convinced that supporting FileMaker 5.5 was important (maybe!).
In any case, let's consider pursuing at least the following 4 additional features:
0. Throw exceptions
1. Option to return fields as arrays (support repeating fields)
2. Option to use POST
3. Option to use ssl
One way, and perhaps the simplest way to support those features would be to add additional arguments, but I worry about that becoming unwieldy. Is there a rule of thumb for the maximum number of arguments for a function? The function would start to look something like this:
simpleFM (string $hostname, string $dbname, string $layoutname [, string $commandstring="-findany" [, string $username [, string $password [, bool $repeating=FALSE [, bool $ssl=FALSE [, string $method="GET" ]]]]])
And arguably might grow to this if we support all the fsockopen() arguments:
simpleFM (string $hostname, string $dbname, string $layoutname [, string $commandstring="-findany" [, string $username [, string $password [, bool $repeating=FALSE [, bool $ssl=FALSE [, string $method="GET"[, int $port [, int &$errno [, string &$errstr [, float $timeout ]]]]]]]]]])
Arguably some of the features above could be achieved by building a sub function with all the arguments, and then making wrapper functions which assume certain arguments, but then it becomes hard to manage different combinations.
Maybe we should consider a simpleFM class so we can break out methods. My main issue with the other classes that exist is all the command methods. By design, I would leave those off if we were to change simpleFM into a class.
So maybe a class is in the offing. Thoughts?
I still think it makes sense to build simpleFMvlists() as a separate function, since it requires totally different parsing logic, so it means a second call to the host, and therefore slows performance even for data-only requests. I generally like to cache value lists anyway, so calling them with a purpose built function suits me just fine.
Jeremiah
single array so that your function signature looks like:
simpleFM (array $connection, string $layoutName, string
$operation="findany" [,array $query_params [,bool $repeating=FALSE]])
$connection = array(
"host" => "server_name",
"port" => "80",
"user" => "username",
"password" => "password",
"ssl" => true,
"databasename" => "filename.fp7"
);
and $query_params would be an array of fieldname=>fieldvalue pairs
$query_params = array (
"fieldname_t" => "value to look");
same thing, but I still can't figure out why the 'databasename' is
considered part of the 'connection' but the 'layoutname' is not. Maybe
I've dealt with so many conversion apps with one layout I use per
FileMaker file. (In fact, in such cases, I often vary the database name
and *don't* vary the layout name, which is usually 'web.' :-) )
This *is* the way that it works with SQL database connections, but SQL
databases rarely have related records in separated 'databases,' just
separate tables.
I agree, using 'query_params' this way is good because the routine can
do the part that encodes the URL characters. It's safer in the long run.
I agree with Thomas on this layout name being part of the 'connection'.