Wednesday, December 16, 2009

Testing

I have been conducting some tests today on different types of datasets, and the first and most troubling thing that I have discovered is on large blocks of search text I have been getting server not responding timeout errors. At this point I dont know if this is a problem with some browser settings on my end or if this is a setting on my webhosts server that is cutting short a process that seems to be taking to long... hummm. One way around this for the industrious, is to install a server on your local machine and ask me for a copy of the source, you will then be able to run the heck out of some large files without the server timing out.

Also it looks like analysis on complicated datasets where primary concepts have many subject compliments there is a tendency for lots of self relation between sets of dependent attributes. This is showing an extremely large amount of intersection concepts. I think that this problem is mainly one of visualization but perhaps it would be beneficial to track polyline paths and display singular routes through the cloud. this may relieve the current sense of confusion when looking at large dataset displays.

Intersection and instructions

I have added an area off of the Pointsware.php page which has some instructions on how to use the software and a description of some of the ideas behind the project.
I also added some function to what gets visualized from the text, now there is a thin magenta line that traces back to any primary concept parents from the set of concepts that were found to be in common between the two query trees.

Also I have done away with the square grid and in its place have allowed the unique concepts to arrange in a cloud, this relieves the tendency for the query tree lines to stack up on top of one another.

I am thinking about maybe looking into a circular presentation of the results but perhaps I should leave it alone and think about the LSA, or writing an interface for the dataset that can run on an Android handheld...

Really what is needed now is for me to spend some time working with creating a lot of different result SVG images and do a little user testing to see where any problems might be showing up. Also I want to look at the different types of data that might be useful to visualize with this type of application, maybe make some test sets and example outputs.

My thought being there is useful information to be found in the network of connections around how we think about the thing we are focusing on.
I cant help but think about the effected range of industry that surrounds automotive manufacturing. There is a nodework of connected buisnesses which have been effected by the loss of jobs due to declining sales in automobiles. each of these buisnesses can be thought of as a line in the dataset where the primary concept is the buisness and the subject compliments are its affiliates, parent companies, products, locations, etc. An analysis of a list of players in the automotive industry might show some interesting dependancies.

All technical analysis aside, I am interested in showing a few examples of Pointsware being used to solve creative problems. After all this whole project stems out of my own desire to understand and systematize the analysis I have observed myself using during fruitful creative studio practice.

Monday, December 14, 2009

Pointsware online is UP!!

I have made some real progress and the interface and query functionality are up and running!
go take a look here

Currently the analyze.php page is still displaying all my debugging output so the SVG is at the bottom of the screen after all of the text, so if you go try it out please scroll to the bottom to see your image.

The SVG looks great courtesy of an iframe tag,
things still left to do in the short term
- highlight linkage path intersections
- make SVG XML file downloadable
- maybe do some tinkering with the display colors

All in all this application has come together faster than I had thought. Which is great because now I can start looking into taking the intersection concepts and providing web resources based off of these terms. I think that the LSA will add a nice level of value for Pointsware users

Very cool...

Tuesday, December 8, 2009

Started work on the web application

I put in place some ground work for the web application today, there is a few new pages on the site and a link to the new interface for Pointsware.
I have been working during this rewrite to take better advantage of nested custom functions to speedup the render rate of the SVG output. Nothing is working yet so that is still to be seen. However I think the code will benefit from the clarity of a more modular use of functions and for anyone else who might want to build on the code I have written the functions will help as well.
once the functions are working I will post some more code and talk a little about the mechanics I am working with.

Friday, December 4, 2009

.SVG is the new black... er .PS

In conjunction with a switch from Ruby to PHP, I am going to make the obvious step from Postscript output to Scalable Vector Graphics format. This makes sense as it will work very well with web application output and is XML based so it is well supported and forward thinking as concerns standardization for the semantic web. (O semantic web, when will you arrive). Writing SVG is a little more complicated than writing PS but you will never have to see the dirty dirty XML needed to create the Pointsware output so lucky you.

However if you are interested in SVG the most useful and technical site I have found with all the specs for the markup can be located at

http://zvon.org/xxl/svgReference/Output/index.html

This site has a comprehensive look at the tags and all their attributes. The W3C has a good overview of SVG as they have for all their compliance standards but the depth of usage for the information is not at the level of the link above.
Also if I might fantasize for a moment... the notion of being able to mashup SVG data from any number of sites around the web to create toolpaths for fabricators makes my rapidprototyping sense tingle. You never know when you might have need to cut google map directions into a pancake with a highpowered router.

Thursday, December 3, 2009

More web app stuff

As I have been working on my website I am beginning to wonder if I should just go ahead and implement the whole Pointsware software in PHP. I keep coming back to this as a way of allowing people to access the mechanisms of the software with ease and without need to install Ruby and go through the whole process of running it from the command line. The implementation as web application is beneficial for access and storage of notes via Internet and mobile devices. Allows for latent semantic analysis to be conducted as an integrated part of the processing and display via easy access to google. Accounts, sharing, and posting of output can also create some community between users. I need to shift my efforts I think... and I have had the nagging suspicion that I need to do an over haul of the basic mechanisms that the code employs anyway, and switching languages would give me this opportunity. However I do think that I will finish implementing the second degree subject compliment linkage paths in the Ruby version and package it all up for people who might be interested in the application as it stands now.

Wednesday, November 25, 2009

Web application

I am not any closer to releasing an alpha version of Poinsware but I have had the opportunity to do some work with PHP.
The possibility of using text output from Pointsware to conduct Google searches that would be filtered by a presedence based on the concepts found in the the compound subject complement output file is becoming conceptualy possible to me.

The assumed benefit of this is once Pointsware has collected the primary concepts and there relations, these words can then be used to conduct searching and latent semantic analysis of the text found withing the search pages to focus on pages that would have a much higher relevance to the users initial Pointsware query. Hopefuly retrieving pages that would have not been found otherwise or not without a large investment in browsing for subject relevance by eyeball.

The use of this tool in a studio setting is aimed at providing its user with quick targeted information with a high level of relevance so that in the midst of the creative process any speculations to possible solutions can be explored.
This is a tool to help users find solutions to questions by collecting useful information of a semantically related nature. This software doesn't provide explicit heuristic ways in which to solve problems, but rather looks to seed the users thinking with information that can be synthesized into a solution.

Friday, November 6, 2009

Close to a first release

I have just successfully implemented the code for outputting second degree subject complements to the .ps file. This next week will be full for me and I dont see myself getting to work on the code or post here. However I am looking forward to getting the code thusfar up on my site. I am in the process of doing an over haul on my website to provide a section for Pointsware releases.

Here is a swatch of code that creates the second degree subject complement matrix point array.
query1att is a variable which holds the list of subject complements for each of the first degree subject complements. This then iterates with a nested loop if there are any subject complements found in the fia array for any of the first degree subject complements.
Please be aware that if you see a "##" it is in place of a double lessthan. The lessthans play havoc on the html tags

query1att.each do |x| #this takes the list of first degree attributes generated from qt1 and itterates through them

fia = fiaconstant #renew the list of fia entries

fpa = fpaconstant #renew the list of fpa entries

thisatt = Array.new #this makes a new array to hold the position info for the current attribute

thisatt##fpa.assoc(x) #this takes the current attribute (x) and searches for it in the position array returning the four part array [name, x, y, listposition]

thisatt = thisatt[0] #this changes this object from a compound array to an array

puts thisatt

sda = Array.new #this makes a new array to hold the posible list of attributes derived from searching the fia array with the current attribute

next if fia.assoc(x) == nil

sda##fia.assoc(x) #this does that search and retrieval

sda = sda[0] #this changes this object from a compound array to an array. important!!

puts "this is x", x

#next if sda[0] == nil #this goes to the next query1att array attribute if there is no entry in fia for the attribute

#next if sda == nil #this goes to the next query1att array attribute if there is no entry in fia for the attribute

sda.each do |att| #this each loop will take the list of possible attributes and itterate through them, this is a nested itterator within the above loop

puts "this is in the sub each loop the current att are", att, thisatt[0].to_s

fpa = fpaconstant #refresh the fpa array

sdaxy = Array.new #a new array to temporaraly hold the current 2nd degree attributes position info

sdaxy##fpa.assoc(att) #this puts that info into that array

sdaxy = sdaxy[0] #these next four statements make vars out of x and y position info from the two itterative arrays so that they can be put into the qt12degatt array for later use in outputting lineto statements in the postscript

next if sdaxy == nil

onex = thisatt[1]

oney = thisatt[2]

twox = sdaxy[1]

twoy = sdaxy[2]

puts onex.to_s, oney.to_s, twox.to_s, twoy.to_s

qt12degatt##[onex.to_s, oney.to_s, twox.to_s, twoy.to_s] #this is the array of xy positions being put into the main output array


end


end


Arrays in Ruby can hold arrays themselves, and this can be useful when you need to have a complex set of indexed data, each position in the super-array holds a sub-array. I have found using these nested arrays is a simple way of creating modular information sets that I can structure data within and in this case run some outputting code on to then write my moveto, lineto statements into the .ps file.

Benefits to Pointsware as a web application

The Sketchup 3d modeling environment available for download over at google, contains an API written for ruby and might make a logical next step as a venue for the visualization of output from Pointsware.
A 3d environment of interconnected concepts configuring themselves based on query terms sounds very compelling. I however am only slightly wooed by the flash of a 3d implementation, and more interested in a migration to a web application.

The application I am envisioning provides the user with a site where they paste a block of compound subject,complement clauses (formatted like the dataset.txt) into a text field and run the code on the website. A .ps file is generated and displayed or downloaded to the users local machine. This way a user has the convince of not being burdened with the installation of ruby and all the subsequent, perhaps novel use of the command line.

This type of web application also provides for hosting of peoples dataset information, editing of stored data through a handheld device, and possible mingling of datas with other people who also have stored datasets. think of a social networking site that conducts latent semantic analysis on your supplied text... how scary... o wait they all do that to you already. (ever wonder why the adds that popup next to your email have relevance to things that you have been emailing about.) (EVERYONE BACK TO PINE!1!!11!!) just kidding.

Out of all the things that a web application would provide the most important to Pointswares use pedagogically is the ability to be accessed remotely via a handheld device. Not for the reason of being able to create analysis from any where, which might be nice. but rather for the ability to record ideas at any point during the day.

Here is the significant relevance, there is a difference between the setting aside of time for the disciplined studio practice of journaling or keeping a sketchbook of ideas, and recording ideas during the course of the day as they occur to you. The difference is composed in large part by the artificial pressure you exert on your brain to come up with or remember ideas that happened recently. The simple solution to this is always have access to some type of record making device. A small pad of paper, a small sketchbook, audio recorder, pda, smart phone anything that is comfortable for your situation to be carried with you at all times.

I have always had trouble getting students to buy into carrying a sketchbook as a daily practice. It is all to easy to feel burdened by an object which needs to be remembered and administrated. Students tell me that it is so hard to remember to take it with them everywhere. To this I love to point out that they all remembered their cell phone.

This is funny, but at the same time useful. Everyone carries a cellphone and most new phones have a notepad feature. (encoding issues withstanding) These devices with their ability to make simple textfiles are the perfect tool for maintaining ideas in the subject,complement clause format used by Pointsware.

It is my opinion and experience that useful ideas are synthesized from contact with daily activity and a current problem your mind is trying to solve. when this synthesis occurs it is best captured immediately or all to often it will flow out of short term memory and be forgotten. My hope for this software system is to harness the wealth of creative effort that a person produces over the course of time and provide a platform for its access.

Thursday, November 5, 2009

Glossary

This post might turn into a glossary in the future but for the time being I will use this to kick around a few definitions for my self

Subject complement: This is a term that I have been using for myself to name the attributes that a primary concept contains / is described by / is an alternate name for / is an attribute of. Dependent attribute is also an analogous term.

Primary Concept: This is the focal point of a list of which its content describes. It is the subject.

Compound Subject Complement Path: This the the path that the software finds which links one primary concept to another primary concept via a series of connections through dependent attributes

Tuesday, November 3, 2009

Writing Postscript output

To start off with this post I should mention a book I have found very useful

Learning PostScript by Doing
By Andre Heck

This book has straightforward examples that build on one another and is easy to use to come quickly up to a usable understanding of postscript coding.
The simplicity and ubiquitous-ness of postscript is its strength as far as this application is concerned. So for ease of use I have decided to use postscript as the output format. Also postscripts human-readable formatting and syntax make for simpler troubleshooting during development.

Postscript can also be read by lots of other softwares and can be easily transformed into formats (such as DXF, and STL) that machines like rapidprototypers and CNC type devices understand. (So for all you computer leaning artists out there learning to code postscript allows you to create files that can drive these machines and fabricate pieces!!)

Here is a snippet of Ruby code that I use to write out some postscript to the output.ps file.

1.upto(qt11degxycount) { |z| z = z - 1

currentlinetoobj = qt11degxy[z]

currentlinetox = currentlinetoobj[0] * xstretch

currentlinetoy = currentlinetoobj[1] * ystretch

makeline = qt1x, " ", qt1y, " lineto"

movetoposition = currentlinetox, " ", currentlinetoy, " moveto"



output.puts "newpath" #writes out a postscript newpath call

output.puts movetoposition.to_s #this writes out the moveto

output.puts "addformathalfcircleqt1" #this changes the formatting to deg 1 qt1 formatting for a half circle

output.puts "newpath" #writes out a postscript newpath call

output.puts "addformat11degline"

output.puts movetoposition.to_s #this writes out the moveto

output.puts makeline.to_s #this writes out the lineto

output.puts " "

}


Another nice thing about the structure of drawing "newpath" elements in a .ps file is their modularity and lack of nesseccary dependancy in their position in the file. In other words you can put a particular "newpath" call in the beginning of the .ps file or all the way at the end and the file will still parse and display. The only real issue with drawing element position in the .ps file is layering. every time you write to the screen it draws over anything that was drawn prior.

The previous code block itterates and produces a set of postscript modules that look like this

newpath
286 1122 moveto
addformathalfcircleqt1
newpath
143 1496 lineto
addformat11degline

Postscript allows for sets of commands to be grouped into a definition and called by the use of a variable. for instance "
addformathalfcircleqt1" is a definition for a set of code that makes a particular marker in the output file. Definitions are a great way to keep formatting under control, and to organize your code.

Here is the definition that is called by the "
addformathalfcircleqt1"

/addformathalfcircleqt1
{
6 0 180 arc
0.2 1 0.1 setrgbcolor %sets color to lime green
2 setlinewidth
stroke
}
def

Notetaking

Quickly a few thoughts on notetaking and the creative process.

One of the main reasons behind my desire to develop this piece of software is I believe short term memory is the place in thinking where conflation, interoperation, synthesizing, and invention happen. Notes on every little remotely interesting thing that occurs to me litter my pockets, desks, hard drives, and everywhere else. These pieces of paper are limited in their usefulness because of their lack of interconnectivity and their difficulty to all be completly reviewed in any expiditious manner. Even a well maintained library of indexed and tidy notebooks eventually becomes to much to look at over time. And while this might be very useful for browsing, it is not capable of doing a targeted creative query to all of the things I have thought over the course of years.

Recall of old yet related thoughts and new relationships into the current mix of questions being asked is a pattern that I have found very fruitful.

Why wait for a specificaly applicable inspiration to strike, when perhapse the most applicable inspiration has struck in the past. And to this end I have become accustomed to a zealous studio practice of recording all possibly interesting thoughs, in the assumption that in the future they might come in handy.(Hmmm, that sounds like some type of brain hoarding, at least this type of hoarding doesnt fillup my closets)


Mutable relationships with flat categorization

I am going to talk a little about the way I have chosen to structure the dataset. The relationships between a primary concept and a dependent attribute only nest by one degree, and this level when interpreted by the software is flattened all the way to 0 levels of categorization. This has a benefit... it may seem abstract but it is a system after all. the short of it is something like this example

fire truck --has--> red --has--> apple

There is a relationship between fire truck and apple through the dependent attribute of red. This simple example of a compounded subject complement illustrates the basic unit of relationship the software I am developing produces. The value of this for the person working in a creative environment is that through the use of organizing relationships with this type of dataset unknown, tacitly known, and forgotten relational possibilities can be exploited.

The dataset is very simply written into a plain text file like this.

fire truck,red,wheels,firemen,truck,vehicle,combustion engine,(etc)

Here the primary concept is the first entry in the comma delimited string. Each line entry in the dataset starts with a newly categorized entry as a primary concept and given dependent attributes.

My intent in structuring the data and the file in this way is to incorporate a highlevel of ease of use, this way once the program is installed it is simple to maintain and note taking in this style is some what intuitive. I have stayed away from a XML scheme for structuring relationships because of its added complexity and the lack of need for more than one level of nested relationship.

Monday, November 2, 2009

An Introduction

This blog will help to give an overview to the use and development of a piece of creativity and thought mapping software I have been working on for the past six months. I feel that it is time to start showing some of the progress I have made and Soon to post alpha version builds of the development software.
I am writing this software in Ruby. I have had a good experience with learning Ruby and would recommend it to others who want to embark on learning a programming language. Ruby is flexible free and there is a lot of documentation on the web and resources to get you through problems and questions. Sites I use constantly are

http://www.rubycentral.com/pickaxe/builtins.html
http://pleac.sourceforge.net/pleac_ruby/
http://www.zenspider.com/Languages/Ruby/QuickRef.html

and here to install ruby itself

http://www.ruby-lang.org/en/

I use Notepad++ for code editing

http://notepad-plus.sourceforge.net/uk/site.htm

and good old fashion Ghostscript, and Imagemagick for making my images from the commandline and visualizing my programs output

http://pages.cs.wisc.edu/~ghost/

http://www.imagemagick.org/script/index.php

All of this stuff is straightforward and can be used by anyone. I dont mean to be patronizing by making this all sound easy, because its isnt. However it is very doable and if you are and artist who wants to incorporate automation, processing and digital visualization into your work, these tools have been very useful for me.