ShinyLight Development » ColdFusion http://www.shinylight.com Web Development and Other Goodness in the World of a Software Engineer. Mon, 09 Jan 2012 00:18:47 +0000 http://wordpress.org/?v=2.9.2 en hourly 1 ErrorDetail I/O Exception: peer not authenticated http://www.shinylight.com/2011/06/08/errordetail-io-exception-peer-not-authenticated/ http://www.shinylight.com/2011/06/08/errordetail-io-exception-peer-not-authenticated/#comments Tue, 07 Jun 2011 23:10:37 +0000 Dan http://www.shinylight.com/?p=1056 Similar to this page, I was getting the following error:

ErrorDetail I/O Exception: peer not authenticated
Filecontent Connection Failure
Mimetype Unable to determine MIME type of file.
Statuscode Connection Failure. Status code unavailable.

Which you get when the JVM has an old version of an SSL certificate. More specifically, I was having problems connecting to ConstantContact, after noticing that they had just renewed their certificate:

Cracking my head open, and trying every solution in the world along what the people posted here, we just decided to reboot the server. This flushed the cache and it worked.

If you can afford to do this, then I recommend it if you want to go for the fastest solution. ConstantContact later provided other possible solutions.

]]> http://www.shinylight.com/2011/06/08/errordetail-io-exception-peer-not-authenticated/feed/ 0 Implementing the Strategy Pattern in ColdFusion http://www.shinylight.com/2010/05/02/implementing-strategy-pattern-coldfusion/ http://www.shinylight.com/2010/05/02/implementing-strategy-pattern-coldfusion/#comments Sun, 02 May 2010 03:27:18 +0000 Dan http://www.shinylight.com/?p=949 In spirit of the amazing book Head First Design Patterns, I wanted to put together a ColdFusion example that depicts the Strategy pattern. Essentially, the Strategy Pattern lets you group related algorithms together so that an object is able to select which algorithm to run at runtime.

So let’s say for example you had a parent class that has two properties and two methods. You now create a subclass that inherits the parent class, meaning that it will gobble up all the properties and methods (whether you like it or not) from the parent class. You can’t choose, for example, what methods you subclass needs, even if some methods don’t make sense for your subclass. It’s an all-or-nothing solution. Yes, you can override, but what if there were 10 methods to override? Also, what if you had to create other types of subclasses – you’ll have to override those as well. Things can get a little sloppy at the end. That’s where the strategy pattern comes in.

With this pattern, you first think about related methods and algorithms. (One method can have various algorithms; different implementations for doing the same thing.) For this blog post’s example, we’ll think of different ways a SuperHero can punch. To keep things simple, let’s give him two ways (two different algorithms for punching) he can punch. He can punch normally, or he can punch you, which freezes you as well. Let’s UML this to make things clearer.

First what the lines mean:


Here’s the UML diagram for our SuperHero scenerio:


From the diagram, you can see that the SuperHero abstract class is able to choose a punch set of algorithms (a strategy). We also see that Punch and PunchFreeze are implementation classes (classes that serve to implement an interface). Both of them have a punch() method that return void (nothing) – in this example, they’ll do stuff, and not return anything.

To make things a little more interesting, and to follow closer the example in the Strategy Pattern chapter of the book, we’re going to also create a subclass called Freezer that inherits the SuperHero class. Also, we’ll create another strategy for kicking. Here’s what the UML for that looks like:


Here’s the CF code:

SuperHero.cfc

<cfcomponent>
 
<!--- This is our abstract class. Responsibility to implement is delegated to --->
<!--- classes that implement interfaces. --->
 
<cffunction name="init" access="public" returntype="SuperHero">  
  <!--- Some other init code goes here.  --->  
  <cfargument name="name" type="string">  
  <cfargument name="gender" type="string">  
  <cfset this.name = arguments.name />  
  <cfset this.gender = arguments.gender />  
  <cfreturn this />  
</cffunction>
 
<cffunction name="setPunchAlgorithm" access="public">
  <!--- The next two lines are key. It's where you set the implementation --->
  <!--- of punch from an object that's being passed in. --->
  <cfargument name="PunchAlgorithm" type="IPunchAlgorithm" required="true" />  
  <cfset this.punchAction = PunchAlgorithm.punch />
</cffunction>
 
<cffunction name="setKickAlgorithm" access="public">
  <cfargument name="KickAlgorithm" type="IKickAlgorithm" required="true" />  
  <cfset this.kickAction = KickAlgorithm.kick />
</cffunction>
 
<!--- This function will be overridden.  --->
<cffunction name="energyProject" access="public" returntype="SuperHero" >  
  You have been pointed at by a weak flash light.  
  <cfreturn this />
</cffunction>
 
</cfcomponent>

IPunchAlgorithm.cfc

<cfinterface>
 
<cffunction name="punch" access="public" />
 
</cfinterface>

Punch.cfc

<cfcomponent implements="IPunchAlgorithm">
 
<cffunction name="punch" access="public">
  You have been punched normally. Ouch.
</cffunction>
 
</cfcomponent>

PunchFreeze.cfc

<cfcomponent implements="IPunchAlgorithm">
 
<cffunction name="punch" access="public">
  You have been punched and are now frozen, stuck. Good luck thawing!
</cffunction>
 
</cfcomponent>

IKickAlgorithm.cfc

<cfinterface>
 
<cffunction name="kick" access="public" />
 
</cfinterface>

Kick.cfc

<cfcomponent implements="IKickAlgorithm">
 
<cffunction name="kick" access="public">
  You have been kicked in the gut. Yummy.
</cffunction>
 
</cfcomponent>

Freezer.cfc

<cfcomponent extends="SuperHero">
 
<!--- We are overriding the energyProject from Freezer's parent class.  --->
<cffunction name="energyProject">
  You have been snowed on. 
</cffunction>  
 
</cfcomponent>

Now let’s actually use these the pattern:

run.cfm

<!--- Create context object.  --->
<cfset IceMan = CreateObject( "component", "SuperHero" ).init( Name = "Iceman", Gender = "Male" ) />
 
<!--- Create a strategy #1 for punching.  --->
<cfset PunchStrategy_1 = CreateObject( "component", "Punch" ) />
 
<!--- Create a different strategy for punching.  --->
<cfset PunchStrategy_2 = CreateObject( "component", "PunchFreeze" ) />
 
<!--- Tell the IceMan object that you'll be using the Punch Strategy #2, NOT #1.  --->
<cfset IceMan.setPunchAlgorithm( PunchStrategy_2 ) />
 
 
<!--- Now let's create a kicking strategy... --->
<cfset KickStrategy = CreateObject( "component", "Kick" ) />
 
<!--- ... and now let's tell the IceMan object that you'll be using the Kick Strategy.  --->
<cfset IceMan.setKickAlgorithm( KickStrategy ) />
 
 
<!--- Now let's see some action!  --->
 
<!--- Punch, using the strategy chosen! --->
<cfset IceMan.punchAction() />
 
<!--- Punch, using the strategy chosen! --->
<cfset IceMan.kickAction() />
 
 
<!--- Let's create another SuperHero object. --->
<cfset Frosty = CreateObject( "component", "SuperHero" ).init( Name = "Frosty the Snowman", Gender = "Unknown" ) />
 
<!--- Let's inspect the objects --->
<p>Notice that both objects have different number of methods - only the methods they need. </p>
<cfdump var="#IceMan#" /><hr />
<cfdump var="#Frosty#" /><hr />
 
 
<!--- Let's create another SuperHero object. --->
<cfset Frostman = CreateObject( "component", "Freezer" ).init( Name = "Calvin Hobbes", Gender = "Male" ) />
<cfset Frostman.energyProject() />

You can download all the CF code here with the original Visio diagram source.

]]> http://www.shinylight.com/2010/05/02/implementing-strategy-pattern-coldfusion/feed/ 0 TSQL Functions Inspired By ColdFusion’s Lists Functions http://www.shinylight.com/2010/04/27/tsql-functions-inspired-by-coldfusions-lists-functions/ http://www.shinylight.com/2010/04/27/tsql-functions-inspired-by-coldfusions-lists-functions/#comments Tue, 27 Apr 2010 00:08:43 +0000 Dan http://www.shinylight.com/?p=944 In my last project, there was a bit of data scrubbing on the database side (SQL Server 2008) that I decided to create a few UDF’s that function similar to ColdFusion’s Lists function. The one that varies a little bit is ListLen(), since I needed to take into account empty tokens. The ChopIf() was inspired by Perl’s chop() function. These UDFs should be SQL Server 2005-compatible.

I should say though, that some of these functions depend on each other. ListLen(), GetToken(), and ChopIf() are independent.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70

------------------------------------------------------------------
-- Functions similarly like ColdFusion ListSort() function,
-- except it currently only sorts strings. 
--
-- Example 1:
--    dbo.ListSort( 'dan is so mega awesome that he rules all the time', 'ASC', ' ' )
--
-- Returns: 
--    all awesome dan he is mega rules so that the time
--
-- Example 2:
--    dbo.ListSort( 'dan is so mega awesome that he rules all the time', 'DESC', ' ' )
--
-- Returns: 
--    time the that so rules mega is he dan awesome all
------------------------------------------------------------------
 
CREATE FUNCTION [dbo].[ListSort]
(
  @string    VARCHAR(2000),  
  @sort_type CHAR(3)       = 'ASC',
  @delimiter VARCHAR(2000) = ','
)
RETURNS VARCHAR(500)
AS
 
BEGIN
 
  DECLARE @position AS INT
  DECLARE @token AS VARCHAR (2000)
  DECLARE @counter   AS INT
  DECLARE @sortedList AS VARCHAR(500)
 
  DECLARE @sortTempTable TABLE ( token VARCHAR(500) )
  DECLARE @sortedTable   TABLE ( token VARCHAR(500) )  
 
  SELECT @string   = @string + @delimiter,
         @counter  = 1,
         @position = 0,
         @token    = ''
 
  WHILE ( PATINDEX( '%' + @delimiter + '%' , @string ) <> 0 ) 
  BEGIN
    SELECT @position = PATINDEX('%' + @delimiter + '%' , @string ),
           @token    = LEFT( @string, @position - 1 ),
           @string   = STUFF( @string, 1, @position, NULL ),
           @counter  = @counter + 1
 
    INSERT @sortTempTable( token ) VALUES( @token )     
  END
 
  SET @sortedList = ''
 
  -- Let's sort the table and put it into @sortedTable
  -- Because of nature of Rank(), we can't set @sortedList in this statement.
  -- Have to separate it into another select clause.
  INSERT INTO @sortedTable
    SELECT LTRIM( token )
    FROM   @sortTempTable
    ORDER  BY CASE WHEN @sort_type = 'ASC'  THEN ( RANK() OVER ( ORDER BY LTRIM(token) ASC ) )
                   WHEN @sort_type = 'DESC' THEN ( RANK() OVER ( ORDER BY LTRIM(token) DESC ) )
              END  
 
  SELECT @sortedList = @sortedList + token + @delimiter
  FROM   @sortedTable
 
  RETURN dbo.ChopIf( @sortedList, @delimiter )
 
END
GO
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

------------------------------------------------------------------
-- Functions sort of like ColdFusion's ListLen() method, but it
-- takes into account empty tokens. 
--
-- Example 1:
--    dbo.ListLen( 'Dan is cool', ' ' )
--
-- Returns: 
--    3
-- 
-- Example 2:
--    dbo.ListLen( 'dan,,very,,,,awesome,', ',' )
--
-- Returns: 
--    8
------------------------------------------------------------------
 
CREATE FUNCTION [dbo].[ListLen]
(
  @string VARCHAR(2000),
  @delimiter VARCHAR(2000) = ','
)
RETURNS INT
AS
BEGIN
 
  DECLARE @loopCount INT, 
          @tokenCount INT
 
  SELECT @loopCount = 0, 
         @tokenCount = 0
 
  -- If it's an empty string, the list length is 0
  IF DATALENGTH( @string ) = 0
    BEGIN
      SET @tokenCount = 0
    END
  ELSE
    BEGIN
      -- Count tokens, including empty ones like dan,,very,,,,awesome,
      SET @tokenCount = @tokenCount + 1
      WHILE ( @loopCount < DATALENGTH( @string ) )
      BEGIN
        IF SUBSTRING( @string, @loopCount, DATALENGTH( @delimiter ) ) = @delimiter
          BEGIN
            SET @tokenCount = @tokenCount + 1
          END
        SET @loopCount = @loopCount + 1
      END
    END
 
  -- Handle extra count from space being delimiter
  IF @delimiter = ' '
    SET @tokenCount = @tokenCount - 1
 
  -- If there's no token to the right of the last delimiter, then count that
  -- as an empty token.
  IF ( RIGHT( @string, 1 ) = @delimiter ) 
  BEGIN
    SET @tokenCount = @tokenCount + 1
  END
 
  RETURN @tokenCount
 
END
GO
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

------------------------------------------------------------------
-- Functions like ColdFusion's ListLast()
-- Gets token value that's been separated by a delimiter.
--
-- Example:
--    dbo.ListLast( 'Dan is cool', ' ' )
--
-- Returns: 
--    cool
------------------------------------------------------------------
 
CREATE FUNCTION [dbo].[ListLast]
(
  @string VARCHAR(2000),
  @delimiter VARCHAR(2000) = ','
)
RETURNS VARCHAR(2000)
AS
BEGIN
 
  RETURN dbo.ListGetAt( @string, dbo.ListLen( @string, @delimiter ) , @delimiter  )
 
END
GO
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

------------------------------------------------------------------
-- Wrapper for GetToken() Function
-- Gets token value that's been separated by a delimiter.
--
-- Example:
--    dbo.ListGetAt( 'Dan is cool', 2, ' ' )
--
-- Returns: 
--    is
------------------------------------------------------------------
 
CREATE FUNCTION [dbo].[ListGetAt]
(
  @string VARCHAR(2000),
  @token INT,
  @delimiter VARCHAR(2000)
)
RETURNS VARCHAR(2000)
AS
BEGIN
  RETURN dbo.GetToken( @string, @token, @delimiter )
END
GO
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

------------------------------------------------------------------
-- Returns the first item in a tokenized list.
--
-- Example:
--    dbo.ListFirst( 'Dan is cool', ' ' )
--
-- Returns: 
--    Dan
------------------------------------------------------------------
 
CREATE FUNCTION [dbo].[ListFirst]
(
  @string VARCHAR(2000),
  @delimiter VARCHAR(2000) = ','
)
RETURNS VARCHAR(2000)
AS
BEGIN
 
  RETURN dbo.ListGetAt( @string, 1, @delimiter )
 
END
GO
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

------------------------------------------------------------------
-- Functions similarly like ColdFusion GetToken() Function.
-- Gets token value that's been separated by a delimiter.
--
-- Example:
--    dbo.GetToken( 'Dan is cool', 2, ' ' )
--
-- Returns: 
--    is
------------------------------------------------------------------
 
CREATE FUNCTION [dbo].[GetToken]
(
  @string VARCHAR(2000),
  @tokenPosition INT,
  @delimiter VARCHAR(2000)
)
RETURNS VARCHAR(2000)
AS
BEGIN
 
  DECLARE @position AS INT
  DECLARE @token AS VARCHAR (2000)
  DECLARE @counter AS INT
 
  SELECT @string = @string + @delimiter,
         @counter = 1,
         @position = 0,
         @token = ''
 
  WHILE ( PATINDEX('%' + @delimiter + '%' , @string ) <> 0) AND ( @tokenPosition + 1 <> @counter )
  BEGIN
    SELECT @position = PATINDEX('%' + @delimiter + '%' , @string),
           @token    = LEFT(@string, @position-1),
           @string   = STUFF(@string, 1, @position, null),
           @counter  = @counter + 1
  END
 
  RETURN @token
END
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

------------------------------------------------------------------
-- Chops the last character if it's @chopped
--
-- Example:
--    dbo.ChopIf( 'Dan is cool!', '!' )
--
-- Returns: 
--    Dan is cool
------------------------------------------------------------------
 
CREATE FUNCTION [dbo].[ChopIf]
(
  @string VARCHAR(2000),
  @chopped VARCHAR(2000)
)
RETURNS VARCHAR(2000)
AS
BEGIN
 
  IF ( RIGHT( @string, DATALENGTH(@chopped) ) = @chopped )
  BEGIN
    SET @string = LEFT( @string, DATALENGTH( @string ) - DATALENGTH( @chopped ) ) 
  END 
 
  RETURN @string
 
END
GO
]]> http://www.shinylight.com/2010/04/27/tsql-functions-inspired-by-coldfusions-lists-functions/feed/ 0 IsNumericList() Function http://www.shinylight.com/2010/04/03/isnumericlist-function/ http://www.shinylight.com/2010/04/03/isnumericlist-function/#comments Fri, 02 Apr 2010 22:59:41 +0000 Dan http://www.shinylight.com/?p=912 Whipped up this UDF while working on a project. It checks to see if the list is valid to use in a TSQL WHERE IN Clause. I wanted to check for a list of valid positive integers (including 0), ignoring spaces.

<cffunction name="isNumericList" returntype="boolean" hint="Check for either single positive number or a set of positive numbers. Spaces ignored." >
 
  <!--- Useful when inserting into an "IN" TSQL list in the WHERE clause.  --->
 
  <cfargument name="list" required="true">
 
  <cfargument name="delimiter" required="false" default=",">   
 
  <cfset var isNumericList = false />
 
  <cfif REFind( "^(\d+)$|^(([\d\s]+#Arguments.delimiter#)+\s*\d+)$", Trim(Arguments.list) ) >
 
    <cfreturn true >
 
  </cfif>
 
  <cfreturn isNumericList >  
 
</cffunction>

Now let’s test it!

<!--- Dummy Data --->
<cfset dataList = ArrayNew(1) />
<cfset dataList[1] = "456c" />
<cfset dataList[2] = "456" />
<cfset dataList[3] = "45c," />
<cfset dataList[4] = "5,,," />
<cfset dataList[5] = "565,651,34,643232,45" />
<cfset dataList[6] = "454,c,45,5454,32" />
<cfset dataList[7] = "121.45,43,565,1,1,2" />
<cfset dataList[8] = "43,54,65,1," />
<cfset dataList[9] = "67,54,73,436," />
<cfset dataList[10] = ",6565,656,77,32,3" />
<cfset dataList[11] = "" />
<cfset dataList[12] = ",43656" />
<cfset dataList[13] = "4365,  55,31,24,   5,   5  ,1,      34" />

Now let’s use it!

<!--- Use it! --->
<cfoutput>
   <cfloop array="#dataList#" index="i">  
      [#i#] => #isNumericList(i)#<br />
   </cfloop>
</cfoutput>

Results!

[456c] => false
 
[456] => true
 
[45c,] => false
 
[5,,,] => false
 
[565,651,34,643232,45] => true
 
[454,c,45,5454,32] => false
 
[121.45,43,565,1,1,2] => false
 
[43,54,65,1,] => false
 
[67,54,73,436,] => false
 
[,6565,656,77,32,3] => false
 
[] => false
 
[,43656] => false
 
[4365,  55,31,24,   5,   5  ,1,      34] => true
]]> http://www.shinylight.com/2010/04/03/isnumericlist-function/feed/ 0 Convert Relative URLs to Absolute http://www.shinylight.com/2010/02/19/convert-relative-urls-to-absolute/ http://www.shinylight.com/2010/02/19/convert-relative-urls-to-absolute/#comments Fri, 19 Feb 2010 04:29:02 +0000 Dan http://www.shinylight.com/?p=874 I put this ColdFusion UDF together the other day to turn relative URLs to Absolute. Code is pretty straightforward.

<cffunction name="URLRelativeToAbsolute" returntype="string"
  hint="Converts relative URLs in an element and converts to absolute. It includes the http:// protocol prefix.">  
 
  <cfargument name="content" type="string" required="true" hint="HTML content that will be scanned and replaced." />
 
  <cfargument name="domain" type="string" required="true" hint="Add domain name to relative links." />
 
  <cfset var local = StructNew() /> 
 
  <!--- The following regexp handles the following elements: link, a, img, script, form, frame. --->
  <cfset local.contentFixed = REReplaceNoCase( Arguments.content, "(href|src|action)=""/?((\./)|(\.\./)+|)(?=[^http])", "\1=""http://" & domain & "/", "all" ) />  
 
  <!--- The following regexp handles the url() attribute of the background CSS property. --->
  <cfset local.contentFixed = REReplaceNoCase( local.contentFixed, "url\((\s)?(')?/?((\./)|(\.\./)+|)(?=[^http])", "url(\2http://" & domain & "/", "all" ) />  
 
  <cfreturn local.contentFixed />    
 
</cffunction>

Usage:

<cfsavecontent variable="htmlContent">
<textarea name="data" rows="20" cols="60">
  <style>
    body { 
      background-image:url('stars.png');
      background-image:url('../stars.png');
      background-image:url('/stars.png');
      background-image:url('/../../../stars.png');
    }
  </style>
  <a href="../../../images/shiny.jpg">Shiny</a>
  <a href="http://www.google.com">This should not be touched</a>
  <img border="0" src="/images/cool.png" /> 
  <link rel="index" href="../../index.asp">
  <form method="POST" action="cgi/processing.cgi"></form>  
</textarea>
</cfsavecontent>
 
<cfoutput>
 
  #htmlContent#
  #URLRelativeToAbsolute( htmlContent, "www.shinylight.com" )#
 
</cfoutput>

Result:

]]> http://www.shinylight.com/2010/02/19/convert-relative-urls-to-absolute/feed/ 0 Read OPML File http://www.shinylight.com/2010/01/03/read-opml-file/ http://www.shinylight.com/2010/01/03/read-opml-file/#comments Sat, 02 Jan 2010 17:26:36 +0000 Dan http://www.shinylight.com/?p=644 Whipped out this little script to read an OPML file from Google Reader. Thought it may be handy.

<cfset GoogleOPMLFile = "C:/google-reader-subscriptions.xml" />
 
<cffile action="READ" variable="xml" file="#GoogleOPMLFile#" /> 
 
<cfset xmlDoc = XMLParse(xml) /> 
 
<cfset StartingDataNode = 2 />
 
<cfset Categories = ArrayLen( xmlDoc.opml.xmlChildren[2].XmlChildren ) />
 
<cfoutput>
 
<cfloop index="i" from="2" to="#Categories#">
 
  <strong>#xmlDoc.opml.xmlChildren[StartingDataNode].XmlChildren[i].XmlAttributes.Title#</strong>
  <ul>
  <cfloop index="j" from="1" to="#ArrayLen( xmlDoc.opml.xmlChildren[StartingDataNode].XmlChildren[i].XmlChildren )#">      	    
    <li>
      <a href="#xmlDoc.opml.xmlChildren[StartingDataNode].XmlChildren[i].XmlChildren[j].XmlAttributes.htmlURL#">
      #xmlDoc.opml.xmlChildren[StartingDataNode].XmlChildren[i].XmlChildren[j].XmlAttributes.title#</a>
    </li>      
  </cfloop>
  </ul>
 
</cfloop>
 
</cfoutput>

The code will display as follows:

]]> http://www.shinylight.com/2010/01/03/read-opml-file/feed/ 0 Asynchronous Upload to Amazon S3 (S3) http://www.shinylight.com/2010/01/01/asynchronous-upload-to-amazon-s3-s3/ http://www.shinylight.com/2010/01/01/asynchronous-upload-to-amazon-s3-s3/#comments Thu, 31 Dec 2009 23:25:50 +0000 Dan http://www.shinylight.com/?p=1115 Requirements

Make sure you have an Amazon S3 Account (get it from tech). JavaScript is mandatory for this to work (to be able to POST to two different domains) upon user submission.

Summary of Problem

This is a summary outlining the solution used in a video uploader. It entailed a form that would leverage the user of our Amazon S3 (S3) account. In addition, because the video files could be large (and to avoid CF limitations), a critical requirement was to upload to that page S3 directly. At the same time, the form field information had to be saved to our database. This meant doing a double post – one to the XYZ domain, and the other to s3.amazon.com. This cross-domain POST could only be done via AJAX.

Here’s visualization:

As you can see, once the user clicks “Submit”, there’s an AJAX HTTP POST to XZY server to save the fields to the database, and then the JavaScript runs a form.submit() on the current form to submit the file via POST to S3.

Introducing Amazon S3

Amazon S3 (Simple Storage Service) is a cloud service whose sole purpose is to store files. To store files into it, one can use its REST API or Admin Web Console (AWS).

Screenshot of the AWS console, with Amazon S3 tab selected.

One gets to the AWS console via an account (can get it from tech) and going to http://aws.amazon.com/s3/ and clicking “Sign in.”

While S3 has many advantages, there are a set of drawbacks as well. To summarize, here’s a list:

Benefits:

  • The max upload size for a file is 5GB. Plenty.
  • For all the times I’ve tested, uploads of different file sizes, everything has gone super smoothly – like butter – so definitely reliable.
  • Amazon is super scalable (as you may already know), so parallel uploading from one or many users is really no problem.
  • Would not affect performance of our servers – there could be many uploads, and they would go fast, without slowing down any other web sites on servers.
  • The speed of the upload is limited to the user’s computer’s specs and internet provider – much faster than our servers.
  • Files can be made secure and unreadable, not just through obscurity – this is sometimes tricky to implement in ColdFusion.

Drawbacks:

To summarize, the reason for some of the drawbacks, is because it’s doing a POST request directly from one domain (us) to another (s3.amazonaws.com). It’s not being channeled through our CF servers.

There are two ways to interact with S3: the REST API, and doing a direct POST. With the REST API, the upload data has to be channeled through a server first before sending to Amazon – this was not what we were looking for, since our servers have issues with large files. So we looked into removing ourselves as the middleman and sending the data directly to S3 – via POST.

Here are the drawbacks, mainly three:

  • If S3 detected an error in the upload, e.g. if the file is too large, there’s no default error page, just a redirect to an XML document hosted on s3.amazonaws.com. There’s no way to set an error page – it’s on Amazon’s to-do list for future release. One can’t even customize the look and feel of the XML document you’re redirected to. Side note: if the upload was successful, it gets redirected to a page you specify (at least there’s some control here).
  • Progress bar reusable code is scare. There’s tons of code out there to do this, however, I could not find one that could cross-domain post. With traditional AJAX, you’re only allowed to do a POST/GET if the URL you’re using is the same domain as the caller page. One could get the code for a progress bar plugin (as there are tons out there) and rewrite it to do a POST and work with Amazon S3 – but that would take a considerate amount of work.
  • Lack of documentation. There’s not enough documentation for handling POST requests in the official Amazon Developer docs, which makes troubleshooting difficult. Doing POST submits is a relatively new feature of Amazon S3, compared to the rest of their APIs.

So the largest hurdle is to code functionality to get around the error page, since some JavaScript magic has to be put in place. That would be another day or so of work just for that, I believe. I already have some code in place that I put together while testing. If we left it as-is, when the user uploads, and if there was an error, the user would see something like this:

Which would, of course, be nonsensical. If the file was too large, they would see a message that the file was too large within the message tags. The user would then have to hit back to return to the form.

We can try another way, probably the easiest. When the user hits submits, it starts showing the animated spinner as it’s uploading. Also, we can tell the user that if he encounters an error page, just hit the back button. Also, keep in mind that there’ll be validation in place before the upload to check for file extension, at the very least. The only edge case to seeing that XML error message is if the file they submitted is over the limit *AND*  they have JavaScript turned off (that overrides the JavaScript file extension validation).

Creating a Basic HTML that POSTs a File to Amazon

Step 1 – Create a Bucket / Folder / Object:
The first thing we need to do a is create a Bucket on S3. To do this the easy way, go to the AWS: https://console.aws.amazon.com/s3/home and create a bucket:

Buckets are where you store your objects (i.e. your files of any format). You can create a folder for further organization:

As you can see here, there are 4 folders here. We can double-click on step1_new_submissions folder and see the objects that are contained within this:

You can right-click on one of those objects (files) and click “Properties”:

Notice that a Properties panel will expand below. To the right, you have three tabs: Details, Permissions, Metadata.

If you click on the Permissions Tab you’ll notice that by default the file that was selected has following permissions set by user USERNAME:

Go back to the details tab and click on the Link:

You’ll notice that you’ll be taken to an XML document in your browser that has the following:

It’s because you have not let it public access. To give it public access, you click on the Permissions tab again, and click “Add more permissions” , set the Grantee to “Everyone” and choose Open/Download and Save.

You can also set the make multiple objects public. Select an object, hold the SHIFT key, then select the other object to select the objects in between. Select “Make Public”:

You can also upload an object manually via the “Upload” button:

Then click “Add more files”

As the file starts uploading, you’ll see the bottom panel show details about the file transfer:

Step 2: Setting up the HTML Form

The S3 REST API is very flexible, as long as you execute the proper method from your application, while at the same time sending the file over from your server to S3 (via a REST method with the correct URI). Traditionally, it would look like this:

Notice how there’s a middle-man that serves as the captor of the data submitted by the form and the file. Then, it sends it long to S3. The middle-man here is crucial. Double the total bandwidth is spent here – the bandwidth to go from the user’s machine to the web server (in this case CF), and then the bandwidth spent transferring the file to S3.

The advantage to this layout is that because the web server acts as a middle-man server, it can modify the data, change its filename, and slice-and-dice anything within the file because the file submitted has to go through it first. Once the middle-man is done, then it sends it to the S3. Drawback is that there’s wasted resources from the middle-man, not to mention there may be limitations on the middle-man to handle large files > 1GB .

As a solution, S3 has a POST method solution where you can POST the file directly to S3:

Setting up the Form tag

Let’s see how we can cross-domain (a domain other than ours) to S3. Rather than doing the following (to post to the same domain):

<form action=”#CGI.SCRIPT_NAME#” method=”post” enctype=”multipart/form-data”>

We do the following:

<form action=”http://s3.amazonaws.com/PastaVideos” method=”post” enctype=”multipart/form-data”>

Where “PastaVideos” is the name of the bucket.

The format of the object URI is as follows:

Step 3: Setting up the Other Form Fields

This is where things get interesting. In order to set up an HTML form that can upload straight to S3, there’s a set of required input fields in the form. They are as follows:


Optional Form Fields

IMPORTANT: If you add any other additional form fields, it will throw an error. If there is in fact a need to add extra form fields, which will be pasted to another server, then you must append the prefix “x-ignore-“. Let’s say for example I have three input fields I want S3 to ignore, then do as follows:

<input type="text" name="x-ignore-lastname" tabindex="2" class="textfield">
<input type="text" name="x-ignore-address1" tabindex="3" class="textfield">
<input type="text" name="x-ignore-address2" tabindex="4" class="textfield">
<input type="text" name="x-ignore-city" tabindex="5" class="textfield">

This is completely legal and will not throw errors.

Grabbing x-ignore- fields in ColdFusion

If you want to grab these form variables via ColdFusion, do something like

Form.x-ignore-lastname

Will not suffice because of the dashes. You’ll have use the bracket/quotes format:

Form[“x-ignore-lastname”]

to grab them.

Also to check for existence or set a default value,

<cfparam name=”form.x-ignore-lastname” default=”parker” />

Or

<cfparam name=”form[“x-ignore-lastname” default=”parker” />

will not work.

You’ll have to use StructKeyexists( Form, “x-ignore-termsagree” ) to check for existence.

HTML Form Example

Putting all variables together from the previous table, we get something something like as follows:

<input type="hidden" name="key" value="step1_new_submissions/9AAAAAAA-D633-0944-9FBCCCCC6CFB161B_${filename}" />  
<input type="hidden" name="acl" value="private" />
<input type="hidden" name="AWSAccessKeyId" value="0N16468ABC47JDAQ2902" />
<input type="hidden" name="policy" value="eyJleHBpcmF0aW9uIjogIjIwMTgtMTAtMjFUMDA6MDA6MDBaIiwKICAiY29uZGl0aW9ucyI6IFsgCiAgICB7ImJ1Y2tldCI6lyZWN0IjogImh0dHA6Ly90ZXN0LXBhc3RhdmlkZW9zLm1pbGxlbm5pdW13ZWIuY29tL3RoYW5rcy5jZm0ifQogIF0KfQ==" />
<input type="hidden" name="signature" value="2AAAA/BhWMg4CCCCC32fzQ=" />
<input type="hidden" name="content-type" value="video/mov" />
<input type="hidden" name="success_action_redirect" value="http://XYZ.com/thanks.cfm" />      
<!--- Ignore All This Stuff... --->
<input type="text" id="x-ignore-firstname" name="x-ignore-firstname" value="peter" />
<input type="text" name="x-ignore-lastname" value="parker" />

Using Amazon’s HTML POST Form Generator

Because setting up the above HTML for the form could be tricky, Amazon has a tool that easily generates the HTML for the above code.

The following is a screenshot of the tool. The form can be found at: http://s3.amazonaws.com/doc/s3-example-code/post/post_sample.html


So the first thing you do is:

1. Fill out the IDs:

Where AWS ID = Access Key ID   and AWS Key = Secret Access Key

2. The next thing we’ll do is fill in the POST URL:

3. The third step is the trickiest step. This is a JSON document that must adhere to the JSON spec. By default, there’s already a default boilerplate JSON document there. Let’s analyze what it means:

Let’s use one a real one from the test-pastavideos.XYZweb.com page:

You’ll notice that it has content-length-range, which checks the max size, in this case being 1 GB, and it also redirects to the index.cfm page when successful.

After you copy and paste that JSON policy, press “Add Policy”. Notice how the fields in the section “Sample Form Based on the Above Fields” has been populated.

4. The fields may look something like this:

5. Now click on Generate HTML:

Note that with the HTML code above, whatever the user uploads will be renamed to “testfile.txt” on S3. To retain the user’s filename, you have to switch it to value=”${filename}”

6. You then copy that generated HTML and paste it into your page. Add any necessary optional fields with the x-ignore-prefixes.

That should give you a basic template for uploading to S3.

NOTE: You cannot change the values of any <input> fields except except the key. More about this in the next section.

Assigning the a unique ID to the object

Keep in mind of these items when assigning a unique filename:

  • You cannot change the user’s filename in JavaScript – once the user selects a file from his computer, you cannot append a GUID because JavaScript will not let you set the value of a file textbox, you can only read.
  • You cannot change append the GUID prefix after the form is submitted, because the filename will go to S3’s server, and there’s no way to run conditional logic once it’s on S3.

To get around these limitations, you generate the GUID or rather a ColdFusion UUID, then append it to your filename. Let’s take a look at an example of the pasta video form:

First let’s show the JSON policy document:

{"expiration": "2018-10-21T00:00:00Z",
  "conditions": 
  [ 
    {"bucket": "PastaVideos"}, 
    ["starts-with", "$key", "step1_new_submissions/"],
    {"acl": "private"},
    ["starts-with", "$Content-Type", "video/mov"],
    ["content-length-range", 0, 10737774],
    {"success_action_redirect": "http://pastavideos.XYZ.com/index.cfm"}
  ]
}

And now the HTML / ColdFusion code:

<cfset Data.videoUUID  = CreateUUID() />        
<form id="videoform" action="http://www.XYZ.com/index.cfm" method="get" enctype="multipart/form-data"> 
 
<!--- Start of Amazon S3 specific variables. --->
<input type="hidden" name="key" value="#Data.key##Data.videoUUIDXX#_${filename}" />   
<input type="hidden" name="acl" value="#Data.acl#" />

In the code above, when the HTML is rendered, it will already have filename S3 will use when it’s finished uploading. Remember, this is not the value of the <input type=”file” /> box.  So the user uploads, it will look like this on the AWS Console:

Now why is the action set to http://www.XYZ.com/index.cfm and method set to get? The HTML has this set, but when the page is loaded, JavaScript immediately runs and changes action to http://s3.amazonaws.com/PastaVideos and method to post:

<cfoutput>
  <!--- Only change the form variables if JavaScript is turned on.  --->  
  $( "##videoform" ).attr( "action", "#Variables.postURL#" ); 
  $( "##videoform" ).attr( "method", "post" );     
</cfoutput>

This is so that if JavaScript is turned off, it doesn’t POST to S3.

Other Sources of Information

Amazon S3 POST Example in HTML

AWS

Helpful Resources

Test Form

Documentation Home Page

Workflow using WS3 policies

Helpful Posts

Asynchronous Upload to Amazon S3 (S3)

Last Updated: 11/29/2010

Author: Dan Romero

Table of Contents

Summary of Problem.. 2

Introducing Amazon S3. 2

Benefits: 3

Drawbacks: 3

Creating a Basic HTML that POSTs a File to Amazon. 5

Step 1 – Create a Bucket / Folder / Object: 5

Step 2: Setting up the HTML Form.. 9

Setting up the Form tag. 10

Step 3: Setting up the Other Form Fields. 11

Optional Form Fields. 13

Grabbing x-ignore- fields in ColdFusion. 13

HTML Form Example. 14

Using Amazon’s HTML POST Form Generator. 14

Assigning the a unique ID to the object. 18

Other Sources of Information. 19

Requirements

Make sure you have an Amazon S3 Account (get it from tech).

JavaScript is mandatory for this to work (to be able to POST to two different domains) upon user
submission.

Summary of Problem

This is a summary outlining the solution used in a video uploader. It entailed a form that would leverage the user of our Amazon S3 (S3) account. In addition, because the video files could be large (and to avoid CF limitations), a critical requirement was to upload to that page S3 directly. At the same time, the form field information had to be saved to our database. This meant doing a double post – one to the XYZ domain, and the other to s3.amazon.com. This cross-domain POST could only be done via AJAX.

Here’s visualization:

As you can see, once the user clicks “Submit”, there’s an AJAX HTTP POST to
XZY server to save the fields to the database, and then the JavaScript runs a
form.submit() on the current form to submit the file via POST to S3.

Introducing Amazon S3

Amazon S3 (Simple Storage Service) is a cloud service whose sole purpose is to store files. To store files into it, one can use its REST API or Admin Web Console (AWS).

Screenshot of the AWS console, with Amazon S3 tab selected.

One gets to the AWS console via an account (can get it from tech) and going to http://aws.amazon.com/s3/ and clicking “Sign in.”

While S3 has many advantages, there are a set of drawbacks as well. To summarize, here’s a list:

Benefits:

The max upload size for a file is 5GB. Plenty.

For all the times I’ve tested, uploads of different file sizes, everything has gone super smoothly – like butter – so definitely reliable.

Amazon is super scalable (as you may already know), so parallel uploading from one or many users is really no problem.

Would not affect performance of our servers – there could be many uploads, and they would go fast, without slowing down any other web sites on servers.

The speed of the upload is limited to the user’s computer’s specs and internet provider – much faster than our servers.

Files can be made secure and unreadable, not just through obscurity – this is sometimes tricky to implement in ColdFusion.

Drawbacks:

To summarize, the reason for some of the drawbacks, is because it’s doing a POST request directly from one domain (us) to another (s3.amazonaws.com). It’s not being channeled through our CF servers.

There are two ways to interact with S3: the REST API, and doing a direct POST. With the REST API, the upload data has to be channeled through a server first before sending to Amazon – this was not what we were looking for, since our servers have issues with large files. So we looked into removing ourselves as the middleman and sending the data directly to S3 – via POST.

Here are the drawbacks, mainly three:

If S3 detected an error in the upload, e.g. if the file is too large, there’s no default error page, just a redirect to an XML document hosted on s3.amazonaws.com. There’s no way to set an error page – it’s on Amazon’s to-do list for future release. One can’t even customize the look and feel of the XML document you’re redirected to. Side note: if the upload was successful, it gets redirected to a page you specify (at least there’s some control here).

Progress bar reusable code is scare. There’s tons of code out there to do this, however, I could not find one that could cross-domain post. With traditional AJAX, you’re only allowed to do a POST/GET if the URL you’re using is the same domain as the caller page. One could get the code for a progress bar plugin (as there are tons out there) and rewrite it to do a POST and work with Amazon S3 – but that would take a considerate amount of work.

Lack of documentation. There’s not enough documentation for handling POST requests in the official Amazon Developer docs, which makes troubleshooting difficult. Doing POST submits is a relatively new feature of Amazon S3, compared to the rest of their APIs.

So the largest hurdle is to code functionality to get around the error page, since some JavaScript magic has to be put in place. That would be another day or so of work just for that, I believe. I already have some code in place that I put together while testing. If we left it as-is, when the user uploads, and if there was an error, the user would see something like this:

Which would, of course, be nonsensical. If the file was too large, they would see a message that the file was too large within the message tags. The user would then have to hit back to return to the form.

We can try another way, probably the easiest. When the user hits submits, it starts showing the animated spinner as it’s uploading. Also, we can tell the user that if he encounters an error page, just hit the back button. Also, keep in mind that there’ll be validation in place before the upload to check for file extension, at the very least. The only edge case to seeing that XML error message is if the file they submitted is over the limit *AND*  they have JavaScript turned off (that overrides the JavaScript file extension validation).

Creating a Basic HTML that POSTs a File to Amazon

Step 1 – Create a Bucket / Folder / Object:
The first thing we need to do a is create a Bucket on S3. To do this the easy way, go to the AWS: https://console.aws.amazon.com/s3/home  and create a bucket:

Buckets are where you store your objects (i.e. your files of any format). You can create a folder for further organization:

As you can see here, there are 4 folders here. We can double-click on step1_new_submissions folder and see the objects that are contained within this:

You can right-click on one of those objects (files) and click “Properties”:

Notice that a Properties panel will expand below. To the right, you have three tabs: Details, Permissions, Metadata.

If you click on the Permissions Tab you’ll notice that by default the file that was selected has following permissions set by user USERNAME:

Go back to the details tab and click on the Link:

You’ll notice that you’ll be taken to an XML document in your browser that has the following:

It’s because you have not let it public access. To give it public access, you click on the Permissions tab again, and click “Add more permissions” , set the Grantee to “Everyone” and choose Open/Download and Save.

You can also set the make multiple objects public. Select an object, hold the SHIFT key, then select the other object to select the objects in between. Select “Make Public”:

You can also upload an object manually via the “Upload” button:

Then click “Add more files”

As the file starts uploading, you’ll see the bottom panel show details about the file transfer:

Step 2: Setting up the HTML Form

The S3 REST API is very flexible, as long as you execute the proper method from your application, while at the same time sending the file over from your server to S3 (via a REST method with the correct URI). Traditionally, it would look like this:

Notice how there’s a middle-man that serves as the captor of the data submitted by the form and the file. Then, it sends it long to S3. The middle-man here is crucial. Double the total bandwidth is spent here – the bandwidth to go from the user’s machine to the web server (in this case CF), and then the bandwidth spent transferring the file to S3.

The advantage to this layout is that because the web server acts as a middle-man server, it can modify the data, change its filename, and slice-and-dice anything within the file because the file submitted has to go through it first. Once the middle-man is done, then it sends it to the S3. Drawback is that there’s wasted resources from the middle-man, not to mention there may be limitations on the middle-man to handle large files > 1GB .

As a solution, S3 has a POST method solution where you can POST the file directly to S3:

Setting up the Form tag

Let’s see how we can cross-domain (a domain other than ours) to S3. Rather than doing the following (to post to the same domain):

<form action=”#CGI.SCRIPT_NAME#” method=”post” enctype=”multipart/form-data”>

We do the following:

<form action=”http://s3.amazonaws.com/PastaVideos” method=”post” enctype=”multipart/form-data”>

Where “PastaVideos” is the name of the bucket.

The format of the object URI is as follows:

Step 3: Setting up the Other Form Fields

This is where things get interesting. In order to set up an HTML form that can upload straight to S3, there’s a set of required input fields in the form. They are as follows:

Tag
Type
Name
Value

input
hidden
key
The location of the object you’ll be uploading. You can consider it as the concatenation of the folder(s) and the filename (don’t use the bucket name):

step1_new_submissions/4059C3_${filename}

The ${filename} is the original filename of the file the user is uploading. The “4059C3_” is a made up ID that is concatenated to the ${filename} that will add uniqueness to the objects in the bucket, if multiple people are uploading to it.

input
hidden
acl
The access control list. Can be set to:

private – Lets the public user upload a file, but not be able to access it once he uploads. To make it accessible, one has to go into the AWS console and change the rights.

public-read – Lets the public user see it after he has uploaded it or anyone else see it.

input
hidden
AWSAccessKeyId
To get this key id, you need to access it by going to:

Then security credentials:

Get the Access Key ID. This id is also called the AWSAccessKeyId.

You should also grab the “Secret Access Key” by clicking on the “Show” on the adjacent column:

Keep the Secret Access Key private! Only the Access Key ID can be made public.

input
hidden
policy
Policy is a Base64 encoded JSON document that outlines the privileges and details of the files being uploaded. More details about this in the next section.

input
hidden
signature
The signature is the policy, HMAC-encrypted using the Secret Access Key.

input
hidden
content-type
The content type is the what kind of mime content the file that will pass through the form will be.

input
hidden
success_action_redirect
This is the URL of where to go when the upload succeeds. It could be any URL.  Also, when redirected, it will add the three additional URL variables:

bucket=PastaVideos

key=step1_new_submissions%2A192DCAE1-D625-0944-9FBCCD5C6CCB161B_ajax5-loader.mov

etag=%22356060aa56ce8955d38ed8c58661497a%22

Optional Form Fields

IMPORTANT: If you add any other additional form fields, it will throw an error. If there is in fact a need to add extra form fields, which will be pasted to another server, then you must append the prefix “x-ignore-“. Let’s say for example I have three input fields I want S3 to ignore, then do as follows:

<input type=”text” name=”x-ignore-lastname” tabindex=”2″ class=”textfield”>

<input type=”text” name=”x-ignore-address1″ tabindex=”3″ class=”textfield”>

<input type=”text” name=”x-ignore-address2″ tabindex=”4″ class=”textfield”>

<input type=”text” name=”x-ignore-city” tabindex=”5″ class=”textfield”>

This is completely legal and will not throw errors.

Grabbing x-ignore- fields in ColdFusion

If you want to grab these form variables via ColdFusion, do something like

Form.x-ignore-lastname

Will not suffice because of the dashes. You’ll have use the bracket/quotes format:

Form[“x-ignore-lastname”]

to grab them.

Also to check for existence or set a default value,

<cfparam name=”form.x-ignore-lastname” default=”parker” />

Or

<cfparam name=”form[“x-ignore-lastname” default=”parker” />

will not work.

You’ll have to use StructKeyexists( Form, "x-ignore-termsagree" ) to check for existence.

HTML Form Example

Putting all variables together from the previous table, we get something something like as follows:

<input type="hidden" name="key" value="step1_new_submissions/9AAAAAAA-D633-0944-9FBCCCCC6CFB161B_${filename}" />

<input type="hidden" name="acl" value="private" />

<input type="hidden" name="AWSAccessKeyId" value="0N16468ABC47JDAQ2902" />

<input type="hidden" name="policy" value="eyJleHBpcmF0aW9uIjogIjIwMTgtMTAtMjFUMDA6MDA6MDBaIiwKICAiY29uZGl0aW9ucyI6IFsgCiAgICB7ImJ1Y2tldCI6lyZWN0IjogImh0dHA6Ly90ZXN0LXBhc3RhdmlkZW9zLm1pbGxlbm5pdW13ZWIuY29tL3RoYW5rcy5jZm0ifQogIF0KfQ==" />

<input type="hidden" name="signature" value="2AAAA/BhWMg4CCCCC32fzQ=" />

<input type="hidden" name="content-type" value="video/mov" />

<input type="hidden" name="success_action_redirect" value="http://XYZ.com/thanks.cfm" />

<!--- Ignore All This Stuff... --->
<input type="text" id="x-ignore-firstname" name="x-ignore-firstname" value="peter" />

<input type="text" name="x-ignore-lastname" value="parker" />

Using Amazon’s HTML POST Form Generator

Because setting up the above HTML for the form could be tricky, Amazon has a tool that easily generates the HTML for the above code.

The following is a screenshot of the tool. The form can be found at: http://s3.amazonaws.com/doc/s3-example-code/post/post_sample.html

So the first thing you do is:

1.       Fill out the IDs:

Where AWS ID = Access Key ID   and AWS Key = Secret Access Key

2.       The next thing we’ll do is fill in the POST URL:

3.       The third step is the trickiest step. This is a JSON document that must adhere to the JSON spec. By default, there’s already a default boilerplate JSON document there. Let’s analyze what it means:

Let’s use one a real one from the test-pastavideos.XYZweb.com page:

You’ll notice that it has content-length-range, which checks the max size, in this case being 1 GB, and it also redirects to the index.cfm page when successful.

After you copy and paste that JSON policy, press “Add Policy”. Notice how the fields in the section “Sample Form Based on the Above Fields” has been populated.

4.       The fields may look something like this:

5.       Now click on Generate HTML:

Note that with the HTML code above, whatever the user uploads will be renamed to “testfile.txt” on S3. To retain the user’s filename, you have to switch it to value=”${filename}”

6.       You then copy that generated HTML and paste it into your page. Add any necessary optional fields with the x-ignore-prefixes.

That should give you a basic template for uploading to S3.

NOTE: You cannot change the values of any <input> fields except except the key. More about this in the next section.

Assigning the a unique ID to the object

Keep in mind of these items when assigning a unique filename:

You cannot change the user’s filename in JavaScript – once the user selects a file from his computer, you cannot append a GUID because JavaScript will not let you set the value of a file textbox, you can only read.

You cannot change append the GUID prefix after the form is submitted, because the filename will go to S3’s server, and there’s no way to run conditional logic once it’s on S3.

To get around these limitations, you generate the GUID or rather a ColdFusion UUID, then append it to your filename. Let’s take a look at an example of the pasta video form:

First let’s show the JSON policy document:

{"expiration": "2018-10-21T00:00:00Z",

"conditions": [

{"bucket": "PastaVideos"},

["starts-with", "$key", "step1_new_submissions/"],

{“acl”: “private”},

["starts-with", "$Content-Type", "video/mov"],

["content-length-range", 0, 10737774],

{“success_action_redirect”: “http://pastavideos.XYZ.com/index.cfm”}

]

}

And now the HTML / ColdFusion code:

<cfset Data.videoUUID  = CreateUUID() />

<form id=”videoform” action=”http://www.XYZ.com/index.cfm” method=”get” enctype=”multipart/form-data”>

<!— Start of Amazon S3 specific variables. —>

<input type=”hidden” name=”key”

value=”#Data.key## Data.videoUUIDXX#_${filename}” />

<input type=”hidden” name=”acl” value=”#Data.acl#” />

In the code above, when the HTML is rendered, it will already have filename S3 will use when it’s finished uploading. Remember, this is not the value of the <input type=”file” /> box.  So the user uploads, it will look like this on the AWS Console:

Now why is the action set to http://www.XYZ.com/index.cfm and method set to get? The HTML has this set, but when the page is loaded, JavaScript immediately runs and changes action to http://s3.amazonaws.com/PastaVideos and method to post:

<cfoutput>

<!— Only change the form variables if JavaScript is turned on.  —>

$( “##videoform” ).attr( “action”, “#Variables.postURL#” );

$( “##videoform” ).attr( “method”, “post” );

</cfoutput>

This is so that if JavaScript is turned off, it doesn’t POST to S3.

Other Sources of Information

Amazon S3 POST Example in HTML

http://aws.amazon.com/code/Amazon%20S3/1093?_encoding=UTF8&jiveRedirect=1

AWS

http://aws.amazon.com/developertools/

Helpful Resources

http://wiki.smartfrog.org/wiki/display/sf/Amazon+S3

http://docs.amazonwebservices.com/AmazonS3/latest/dev/index.html?AccessPolicyLanguage_UseCases_s3_a.html

http://docs.amazonwebservices.com/AmazonS3/latest/gsg/

http://docs.amazonwebservices.com/AmazonS3/2006-03-01/dev/index.html?UsingHTTPPOST.html

http://docs.amazonwebservices.com/AmazonS3/2006-03-01/dev/index.html?HTTPPOSTExamples.html

http://docs.amazonwebservices.com/AmazonS3/2006-03-01/dev/index.html?Introduction.html#S3_ACLs

Test Form

http://s3.amazonaws.com/doc/s3-example-code/post/post_sample.html

Documentation Home Page

http://aws.amazon.com/documentation/s3/

Workflow using WS3 policies

http://docs.amazonwebservices.com/AmazonS3/latest/dev/

Helpful Posts

http://aws.amazon.com/articles/1434?_encoding=UTF8&jiveRedirect=1

https://forums.aws.amazon.com/message.jspa?messageID=89017

https://forums.aws.amazon.com/message.jspa?messageID=88188

]]> http://www.shinylight.com/2010/01/01/asynchronous-upload-to-amazon-s3-s3/feed/ 0 yUML and ColdFusion http://www.shinylight.com/2009/12/29/yuml-and-coldfusion/ http://www.shinylight.com/2009/12/29/yuml-and-coldfusion/#comments Tue, 29 Dec 2009 06:45:15 +0000 Dan http://www.shinylight.com/?p=594 I just tried to write a quick script in Python that scans CFCs and generates a yUML URL to diagram. I pointed my script to my root CFC path and I got a 13K strlen URL. I pasted it in the address bar to see what happened and I got the following:

Request-URI Too Large
 
The requested URL's length exceeds the capacity limit for this server.
Apache/2.2.3 (Debian) Phusion_Passenger/2.0.2 Server at Ess000235.gtcust.grouptelecom.net Port 80

I wonder what the limitation is. I suppose I’ll have to do a CFC per diagram and then bind them together somehow. I’m choosing Python so this script can be part of my build script.

Here’s the code so far, which of course, could be optimized:

import re
import os
 
# UML Syntax
# http://yuml.me/diagram/class/[User|Property1;Property2|Method1();Method2()]
# http://yuml.me/diagram/class/
# [
#   User
#   |
#     Property1;
#     Property2
#   |
#     Method1();
#     Method2()
#  ]
 
 
# Master Path
ROOT_PATH = 'C:\\temp\\cf-yuml'
 
def SearchForFile( rootpath, searchfor, includepath = 0 ):
 
  # Search for a file recursively from a root directory.
  #  rootpath  = root directory to start searching from.
  #  searchfor = regexp to search for, e.g.:
  #                 search for *.jpg : \.exe$                     
  #  includepath = appends the full path to the file
  #                this attribute is optional
  # Returns a list of filenames that can be used to loop
  # through.
  #
  # TODO: Use the glob module instead. Could be faster.  
  names = []
  append = ""
  for root, dirs, files in os.walk( rootpath ): 
    for name in files:
      if re.search( searchfor, name ):
        if includepath == 0:
          root = ""          
        else:          
          append = "\\"
        names.append( root + append + name )        
  return names  
 
 
def getCFCInfo ( FILE, path ):
  FILE.seek( 0, 0 )  
  CFCLines = FILE.readlines()
 
  CFCFunctions  = []
  CFCProperties = []
  CFC           = {}
 
  for i in CFCLines:
    # Get names of methods  
    if re.search( "^<cffunction", i , re.IGNORECASE | re.MULTILINE ):    
      CFCFunctions.append( re.search( r'name\s*=\s*"([\w$-]+)"', i, re.DOTALL | re.IGNORECASE).group(1) )
 
  # Get names of properties
    if re.search( "^<cfproperty", i , re.IGNORECASE | re.MULTILINE ):    
      CFCProperties.append( re.search( r'name\s*=\s*"([\w$-]+)"', i, re.DOTALL | re.IGNORECASE).group(1) )     
 
  CFC = { "properties":CFCProperties, "methods":CFCFunctions }  
 
  # Generate URL
  strFunctions  = ""
  strProperties = ""
 
  for i in CFCFunctions:
    strFunctions  += i + "();"
 
  for i in CFCProperties:
    strProperties += i + ";"  
 
  CFCFileName = re.search(r"\\([\w-]+)\.cfc$", path, re.DOTALL | re.IGNORECASE).group(1)  
  return "[" + CFCFileName + "|" + ( strProperties.strip()[:-1] + "|" if strProperties.strip()[:-1] else "" ) + strFunctions.strip()[:-1] + "]"  
 
URL = ""
 
for i in SearchForFile( ROOT_PATH, "\.cfc$", 1 ):
  CFCFile = open( i, "r" )
  URL += getCFCInfo( CFCFile, i ) + ","
  CFCFile.close()
 
URL = URL[:-1]
print "http://yuml.me/diagram/class/" + URL

I’ll keep working on this as time goes on. So far it just goes through all the CFC’s from the path you point to. It will crawl through all sub directories. There’s no relationship between classes, however. Not yet at least.

]]> http://www.shinylight.com/2009/12/29/yuml-and-coldfusion/feed/ 0