=== modified file 'CHANGES' --- CHANGES 2007-11-18 14:59:06 +0000 +++ CHANGES 2010-10-19 19:11:10 +0000 @@ -1,6 +1,94 @@ -Please initial your changes (there's a key at bottom) and add a date for each -release -================================================================================ +2.4.2 (February 8th, 2010) + - Fix issue where subclasses of Template failed to pick up attributes in the + searchlist + - Remove old/outdated bundled memcached python client + - Allow for #encoding directives to exist after a comment (i.e. not the first + line in a module) + - Remove support for WebWare servlets (which caused significant performance + slowdowns on Mac OS X) + - Old/stale code pruned in preparation for Python 3 support + +2.4.1 (December 19th, 2009) + - --quiet flag added to `cheetah` to silence printing to stdout (abbeyj) + - Refactoring to minimize the amount of forked code for Python3 (rtyler) + - Template.compile() will no longer create class names with numerous leading + underscores (rtyler; reported by Kirill Uhanov) + - DirectiveAnalyzer (cheetah-analyze script) added to report directive usage in templates (rtyler) + - Older LaTeX docs converted to rst for Sphinx (rtyler) + - Prevent #raw blocks from evaluating $-placeholders and escaped strings (karmix0) + - New tests added to verify PSP behavior and other untested internals (rtyler) + +2.4.0 (October 24th, 2009) + - Fix a major performance regression in Template.__init__() + - More graceful handling of unicode when calling .respond() to render a template + - Minor code updates + - Update the default filter (thanks mikeb!) + +2.3.0 (October 24th, 2009) (loosely equivalent to 2.4.0) + - Fix a major performance regression in Template.__init__() + - More graceful handling of unicode when calling .respond() to render a template + - Minor code updates + - Update the default filter (thanks mikeb!) + +2.2.2 (September 10th, 2009) + - Prevent _namemapper.c from segfaulting when PyImport_ImportModule fails for some reason (Bogdano Arendartchuk ) + - Removal of the contrib/markdown module (in favor of a setuptools dependency) + - Default setup.py to use setuptools by default, failing that, fall back to distutils + - Improvements to setup.py to support building for Windows (thanks abbeyj!) + - Improvements to C-based NameMapper for Windows + - Fixes for a swath of unit tests on Windows + - Re-enabling the EOL tests (whoops) + - Fix for unicode/utf-8 dynamic compilation error (thanks mikeb!) (Test.Unicode.JBQ_UTF8_Test8) + - 0000010: [Templates] Failure to execute templates on Google App Engine (rtyler) + - 0000026: [Compiler] Support multiple inheritance (rtyler) + + +2.2.1 (June 1st, 2009) + - 0000020: [Templates] Builtin support for using Cheetah with Django (rtyler) + - 0000021: [Compiler] @static and @classmethod don't properly define the _filter local (rtyler) + - 0000023: [Compiler] Update Template super calls to use super() (rtyler) + - Update all references to communitycheetah.org to point back at cheetahtemplate.org + +2.2.0 (May 17th, 2009) + - Switch all internal representations of template code to unicode objects instead of str() objects + - Convert unicode compiled template to an utf8 char buffer when writing to a file (Jean-Baptiste Quenot ) + - 0000011: [Templates] Calling a function with arguments calls the function with None (rtyler) + - 0000015: [Tests] Resolve test failures in 'next' branch (rtyler) + - 0000019: [Templates] Properly warn when joining unicode and non-unicode objects in DummyTransaction (rtyler) + +2.1.2 (May 5, 2009) + - 0000006: [Templates] Support @staticmethod and @classmethod (rtyler) + +2.1.1 (April 16, 2009) + - Support __eq__() and __ne__() the way you might expect in src/Tools/RecursiveNull (patch suggested by Peter Warasin ) + - Applied patch to avoid hitting the filesystem to get the file modification time everytime a #include directive is processed (Jean-Baptiste Quenot ) + - Applied patch to fix some annoying cases when Cheetah writes to stderr instead of propagating the exception (Jean-Baptiste Quenot ) + - Added KDE editor support + - Applied patch to correct importHook behavior on Python 2.6 (reported/patched by Toshio Ernie Kuratomi ) + - Correct unicode issue when calling/embedding unicode templates inside of other templtes (testcase Tests.Unicode.JPQ_UTF8_Test3. reported by Jean-Baptiste Quenot ) + - Added --shbang option (e.g. "cheetah compile --shbang '#!/usr/bin/python2.6' ") + - Removed dependency on optik OptionParser in favor of builtin Python optparse module + - Introduction of the #transform directive for whole-document filtering + - Introduction of Cheetah.contrib.markdown and Cheetah.Filters.Markdown for outputting a markdown processed template (meant for #transform) + - Cheetah.Filters.CodeHighlighter, pygments-based code highlighting filter for use with #transform + - Addition of "useLegacyImportMode" compiler setting (defaulted to True) to allow for older (read: broken) import behavior + +2.1.0.1 (March 27, 2009) + - Fix inline import issue introduced in v2.1.0 + +2.1.0 (March 16, 2009) + - Quiet DeprecationWarnings being printed to stderr when using Cheetah on Python 2.6 and up. Patch suggested by Satoru SATOH + - Apply patch to support parallel compilation of templates courtesy of Evan Klitzke + - Corrected issue when __getattr__ calls on searchList objects raise exceptions (tyler@slide.com) + - make autocalling in valueForName correctly ignore newstyle classes and instances + that are callable, as it does for oldstyle classes and instances. Patch + from lucas@endian.com + [TR] + - made it possible to chain multiple decorators to a method #def [TR with + patch from Graham Dennis] + - fixed a bug in _eatMultiLineDef that Graham Dennis reported. [TR] + - fixed 'module.__init__() argument 1 must be string, not unicode' bug in + Template.py reported by Erwin Ambrosch [TR] 2.0.1 (Nov 16, 2007) - fixed a deadlock Christoph Zwerschke found in Cheetah.ImportHooks. === added directory 'Cheetah.egg-info' === added file 'Cheetah.egg-info/PKG-INFO' --- Cheetah.egg-info/PKG-INFO 1970-01-01 00:00:00 +0000 +++ Cheetah.egg-info/PKG-INFO 2010-10-19 19:11:10 +0000 @@ -0,0 +1,47 @@ +Metadata-Version: 1.0 +Name: Cheetah +Version: 2.4.3 +Summary: Cheetah is a template engine and code generation tool. +Home-page: http://www.cheetahtemplate.org/ +Author: R. Tyler Ballance +Author-email: cheetahtemplate-discuss@lists.sf.net +License: UNKNOWN +Description: Cheetah is an open source template engine and code generation tool. + + It can be used standalone or combined with other tools and frameworks. Web + development is its principle use, but Cheetah is very flexible and is also being + used to generate C++ game code, Java, sql, form emails and even Python code. + + Documentation + ================================================================================ + For a high-level introduction to Cheetah please refer to the User's Guide + at http://www.cheetahtemplate.org/learn.html + + Mailing list + ================================================================================ + cheetahtemplate-discuss@lists.sourceforge.net + Subscribe at http://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss + + Credits + ================================================================================ + http://www.cheetahtemplate.org/credits.html + + Recent Changes + ================================================================================ + See http://www.cheetahtemplate.org/CHANGES.txt for full details + + +Platform: UNKNOWN +Classifier: Development Status :: 5 - Production/Stable +Classifier: Intended Audience :: Developers +Classifier: Intended Audience :: System Administrators +Classifier: License :: OSI Approved :: MIT License +Classifier: Operating System :: OS Independent +Classifier: Programming Language :: Python +Classifier: Topic :: Internet :: WWW/HTTP +Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content +Classifier: Topic :: Internet :: WWW/HTTP :: Site Management +Classifier: Topic :: Software Development :: Code Generators +Classifier: Topic :: Software Development :: Libraries :: Python Modules +Classifier: Topic :: Software Development :: User Interfaces +Classifier: Topic :: Text Processing === added file 'Cheetah.egg-info/SOURCES.txt' --- Cheetah.egg-info/SOURCES.txt 1970-01-01 00:00:00 +0000 +++ Cheetah.egg-info/SOURCES.txt 2010-10-19 19:11:10 +0000 @@ -0,0 +1,80 @@ +CHANGES +LICENSE +MANIFEST.in +README.markdown +SetupConfig.py +SetupTools.py +TODO +setup.py +Cheetah.egg-info/PKG-INFO +Cheetah.egg-info/SOURCES.txt +Cheetah.egg-info/dependency_links.txt +Cheetah.egg-info/requires.txt +Cheetah.egg-info/top_level.txt +bin/cheetah +bin/cheetah-analyze +bin/cheetah-compile +cheetah/CacheRegion.py +cheetah/CacheStore.py +cheetah/CheetahWrapper.py +cheetah/Compiler.py +cheetah/DirectiveAnalyzer.py +cheetah/Django.py +cheetah/DummyTransaction.py +cheetah/ErrorCatchers.py +cheetah/FileUtils.py +cheetah/Filters.py +cheetah/ImportHooks.py +cheetah/ImportManager.py +cheetah/NameMapper.py +cheetah/Parser.py +cheetah/Servlet.py +cheetah/SettingsManager.py +cheetah/SourceReader.py +cheetah/Template.py +cheetah/TemplateCmdLineIface.py +cheetah/Unspecified.py +cheetah/Version.py +cheetah/__init__.py +cheetah/convertTmplPathToModuleName.py +cheetah/Macros/I18n.py +cheetah/Macros/__init__.py +cheetah/Templates/SkeletonPage.py +cheetah/Templates/SkeletonPage.tmpl +cheetah/Templates/_SkeletonPage.py +cheetah/Templates/__init__.py +cheetah/Tests/Analyzer.py +cheetah/Tests/CheetahWrapper.py +cheetah/Tests/Cheps.py +cheetah/Tests/Filters.py +cheetah/Tests/Misc.py +cheetah/Tests/NameMapper.py +cheetah/Tests/Parser.py +cheetah/Tests/Performance.py +cheetah/Tests/Regressions.py +cheetah/Tests/SyntaxAndOutput.py +cheetah/Tests/Template.py +cheetah/Tests/Test.py +cheetah/Tests/Unicode.py +cheetah/Tests/__init__.py +cheetah/Tests/xmlrunner.py +cheetah/Tools/CGITemplate.py +cheetah/Tools/MondoReport.py +cheetah/Tools/MondoReportDoc.txt +cheetah/Tools/RecursiveNull.py +cheetah/Tools/SiteHierarchy.py +cheetah/Tools/__init__.py +cheetah/Tools/turbocheetah/__init__.py +cheetah/Tools/turbocheetah/cheetahsupport.py +cheetah/Tools/turbocheetah/tests/__init__.py +cheetah/Tools/turbocheetah/tests/test_template.py +cheetah/Utils/Indenter.py +cheetah/Utils/Misc.py +cheetah/Utils/WebInputMixin.py +cheetah/Utils/__init__.py +cheetah/Utils/htmlDecode.py +cheetah/Utils/htmlEncode.py +cheetah/Utils/statprof.py +cheetah/c/Cheetah.h +cheetah/c/_namemapper.c +cheetah/c/cheetah.h \ No newline at end of file === added file 'Cheetah.egg-info/dependency_links.txt' --- Cheetah.egg-info/dependency_links.txt 1970-01-01 00:00:00 +0000 +++ Cheetah.egg-info/dependency_links.txt 2010-10-19 19:11:10 +0000 @@ -0,0 +1,1 @@ + === added file 'Cheetah.egg-info/requires.txt' --- Cheetah.egg-info/requires.txt 1970-01-01 00:00:00 +0000 +++ Cheetah.egg-info/requires.txt 2010-10-19 19:11:10 +0000 @@ -0,0 +1,1 @@ +Markdown >= 2.0.1 \ No newline at end of file === added file 'Cheetah.egg-info/top_level.txt' --- Cheetah.egg-info/top_level.txt 1970-01-01 00:00:00 +0000 +++ Cheetah.egg-info/top_level.txt 2010-10-19 19:11:10 +0000 @@ -0,0 +1,1 @@ +Cheetah === modified file 'MANIFEST.in' --- MANIFEST.in 2006-07-26 22:03:15 +0000 +++ MANIFEST.in 2010-10-19 19:11:10 +0000 @@ -1,7 +1,7 @@ -include MANIFEST.in *.py *.cfg TODO CHANGES LICENSE README examples docs bin -recursive-include src *.py *.tmpl *.txt +include MANIFEST.in *.py *.cfg TODO CHANGES LICENSE README.markdown examples docs bin +recursive-include cheetah *.py *.tmpl *.txt *.h recursive-include bin * recursive-include docs * recursive-include examples * -recursive-exclude src *.pyc *~ *.aux +recursive-exclude cheetah *.pyc *~ *.aux recursive-exclude docs *~ *.aux === modified file 'PKG-INFO' --- PKG-INFO 2007-11-18 14:59:06 +0000 +++ PKG-INFO 2010-10-19 19:11:10 +0000 @@ -1,9 +1,9 @@ Metadata-Version: 1.0 Name: Cheetah -Version: 2.0.1 +Version: 2.4.3 Summary: Cheetah is a template engine and code generation tool. -Home-page: http://www.CheetahTemplate.org/ -Author: Tavis Rudd +Home-page: http://www.cheetahtemplate.org/ +Author: R. Tyler Ballance Author-email: cheetahtemplate-discuss@lists.sf.net License: UNKNOWN Description: Cheetah is an open source template engine and code generation tool. @@ -15,7 +15,7 @@ Documentation ================================================================================ For a high-level introduction to Cheetah please refer to the User's Guide - at http://cheetahtemplate.org/learn.html + at http://www.cheetahtemplate.org/learn.html Mailing list ================================================================================ @@ -24,869 +24,12 @@ Credits ================================================================================ - http://cheetahtemplate.org/credits.html - - Praise - ================================================================================ - "I'm enamored with Cheetah" - Sam Ruby, senior member of IBM Emerging - Technologies Group & director of Apache Software Foundation - - "Give Cheetah a try. You won't regret it. ... Cheetah is a truly powerful - system. ... Cheetah is a serious contender for the 'best of breed' Python - templating." - Alex Martelli - - "People with a strong PHP background absolutely love Cheetah for being Smarty, - but much, much better." - Marek Baczynski - - "I am using Smarty and I know it very well, but compiled Cheetah Templates with - its inheritance approach is much powerful and easier to use than Smarty." - - Jaroslaw Zabiello - - "There is no better solution than Cheetah" - Wilk - - "A cheetah template can inherit from a python class, or a cheetah template, and - a Python class can inherit from a cheetah template. This brings the full power - of OO programming facilities to the templating system, and simply blows away - other templating systems" - Mike Meyer - - "Cheetah has successfully been introduced as a replacement for the overweight - XSL Templates for code generation. Despite the power of XSL (and notably XPath - expressions), code generation is better suited to Cheetah as templates are much - easier to implement and manage." - The FEAR development team - (http://fear.sourceforge.net/docs/latest/guide/Build.html#id2550573) - - "I've used Cheetah quite a bit and it's a very good package" - Kevin Dangoor, - lead developer of TurboGears. + http://www.cheetahtemplate.org/credits.html Recent Changes ================================================================================ - See http://cheetahtemplate.org/docs/CHANGES for full details. - - Please initial your changes (there's a key at bottom) and add a date for each - release - ================================================================================ - - 2.0.1 (Nov 16, 2007) - - fixed a deadlock Christoph Zwerschke found in Cheetah.ImportHooks. - [TR] - - 2.0 (Oct 12, 2007) - !!!THIS RELEASE REQUIRES RECOMPILATION OF ALL COMPILED CHEETAH TEMPLATES!!! - - - fixed exception handling issue in the C implemenation of NameMapper - [patch from Eric Huss] - - - fixed filtering of #included subtemplates - [patch from Brian Bird] - - See the release notes from 2.0b1-5 and 2.0rc1-8 for other changes since - Cheetah 1.0. - - - 2.0rc8 (April 11, 2007) - !!!THIS RELEASE REQUIRES RECOMPILATION OF ALL COMPILED CHEETAH TEMPLATES!!! - Core Changes: [TR] - - - added a '#unicode ' directive to indicate that the output of the - template should be a unicode string even if the template source is a - normal byte string. - - - #unicode and #encoding are mutually exclusive. Use one or the other. - - #unicode must be on a line by itself. - - Strings in embedded code must be explictly marked as unicode if they - contain non-ascii chars: - - #unicode latin-1 - $f(u"") ## right - $f("") ## wrong - - However, this works fine: - - #unicode latin-1 - blah blah blah blah - - - fixed several unicode bugs in the compiler. - - - fixed some unicode issues in the standard filters. - - - fixed a few minor bugs in code that never gets called. Thanks to - Alejandro Dubrovsky for pointing them out. - - - make RawOrEncodedUnicode the baseclass of all filters and remove some - unused/redudant filters - - - added new compiler setting 'addTimestampsToCompilerOutput'. See Brian - Bird's post about it. He stores his cheetah generated .py files in - subversion and needed to disable the timestamp code so svn wouldn't care - when he recompiles those .py modules. - - - added the #super directive, which calls the method from the parent class - which has the same as the current #def or #block method. - - #def foo - ... child output - #super ## includes output of super(, self).foo() - ... child output - #end def - - - #def bar(arg) - ... child output - #super(arg) ## includes output of super(, self).bar(arg) - ... child output - #end def - - - added some unit tests for the new directives - - - 2.0rc7 (July 4, 2006) - !!!THIS RELEASE REQUIRES RECOMPILATION OF ALL COMPILED CHEETAH TEMPLATES!!! - Core Changes: [TR] - - extended the #implements directive so an arguments list can be declared in - the same fashion as #def and #block. - - - made the parser raise ParseError when $*placeholder, $*5*placeholder, - $(placeholder), etc. are found within expressions. They are only valid in - top-level text. - - - tweaked the parser so it's possible to place a comment on the same line as - a directive without needing to explicitly close the directive first. This - works regardless of whether or not you added a colon. - - self.verify("#if 1:\n$aStr\n#end if\n", - "blarg\n") - - self.verify("#if 1: \n$aStr\n#end if\n", - "blarg\n") - - self.verify("#if 1: ##comment \n$aStr\n#end if\n", - "blarg\n") - - self.verify("#if 1 ##comment \n$aStr\n#end if\n", - "blarg\n") - - Previously, that last test would have required an extra # to close the #if - directive before the comment directive started: - self.verify("#if 1 ###comment \n$aStr\n#end if\n", - "blarg\n") - - Code that makes use of explicit directive close tokens immediately followed by - another directive will still work as expected: - #if test##for i in range(10)# foo $i#end for##end if - - - safer handling of the baseclass arg to Template.compile(). It now does - the right thing if the user passes in an instance rather than a class. - - ImportHooks: [TR] - - made it possible to specify a list of template filename extentions that are - looped through while searching for template modules. E.g.: - import Cheetah.ImportHooks - Cheetah.ImportHooks.install(templateFileExtensions=('.tmpl','.cheetah')) - - Core changes by MO: - - Filters are now new-style classes. - - WebSafe and the other optional filters in Filters.py now use - RawOrEncodedUnicode instead of Filter as a base class. This allows them - to work with Unicode values containing non-ASCII characters. - User-written custom filters should inherit from - RawOrEncodedUnicode and call the superclass .filter() instead of str(). - str() as of Python 2.4.2 still converts Unicode to string using - ASCII codec, which raises UnicodeEncodeError if it contains non-ASCII - characters. - - 2.0rc6 (Feb 4, 2006) - !!!THIS RELEASE REQUIRES RECOMPILATION OF ALL COMPILED CHEETAH TEMPLATES!!! - Core Changes: [TR] - - added a Cheetah version dependency check that raises an assertion if a - template was compiled with a previous version of Cheetah whose templates - must be recompiled. - - - made the Cheetah compilation metadata accessible via class attributes in - addition to module globals - - - major improvement to exception reporting in cases where bad Python syntax - slips past the Cheetah parser: - """ - File "/usr/lib/python2.4/site-packages/Cheetah/Template.py", line 792, in compile - raise parseError - Cheetah.Parser.ParseError: - - Error in the Python code which Cheetah generated for this template: - ================================================================================ - - invalid syntax (DynamicallyCompiledCheetahTemplate.py, line 86) - - Line|Python Code - ----|------------------------------------------------------------- - 84 | - 85 | write('\n\n') - 86 | for i an range(10): # generated from line 4, col 1 - ^ - 87 | _v = i # '$i' on line 5, col 3 - 88 | if _v is not None: write(_filter(_v, rawExpr='$i')) # from line 5, col 3. - 89 | write('\n') - - ================================================================================ - - Here is the corresponding Cheetah code: - - Line 4, column 1 - - Line|Cheetah Code - ----|------------------------------------------------------------- - 2 |#compiler useNameMapper=False - 3 | - 4 |#for i an range(10) - ^ - 5 | $i - 6 |#end for - 7 | - """ - - 2.0rc5 (Feb 3, 2006) - !!!THIS RELEASE REQUIRES RECOMPILATION OF ALL COMPILED CHEETAH TEMPLATES!!! - Core Changes: [TR] - - fixed a memory leak in Template.compile(), reported by Andrea Arcangeli - - simplified concurrency locking and compile caching in Template.compile() - - The command line tool (CheetahWrapper.py): - - added new option --settings for supplying compiler settings - - added new option --templateAPIClass to replace the environment var - CHEETAH_TEMPLATE_CLASS lookup I added in 2.0b1 - - 2.0rc4 (Jan 31, 2006) - !!!THIS RELEASE REQUIRES RECOMPILATION OF ALL COMPILED CHEETAH TEMPLATES!!! - Core Changes: [TR] - - fixed a typo-bug in the compile hashing code in Template.compile() - - improved the macros framework and made it possible to implement macros in - Python code so they can be shared between templates - - more work on the #i18n directive. It's now a macro directive. - - added new Cheetah.Macros package - - more tests - - 2.0rc3 (Jan 29, 2006) - !!!THIS RELEASE REQUIRES RECOMPILATION OF ALL COMPILED CHEETAH TEMPLATES!!! - Core Changes: [TR] - - added short-form single line versions of all directives that have an #end - tag, except for #errorCatcher: - #if, #else, #elif, #unless, - #for, #while, #repeat, - #try, #except, #finally, - #cache, #raw - #call, #capture - - The #def and #block directives already had single-line versions. - #if cond: foo - #elif cond2: bar - #else: blarg - - #for i, val in enumerate(vals): $i-$val - - Note that if you accidentally leave a colon at the end of one of these - directives but nothing else follows it, aside from whitespace, the parser - will treat it as a normal multi-line directive. - - The first leading space after the colon is discarded. Any additional - spaces will be included in the output. - - Also note, if you use the short form versions of #if/#else/#elif you must - it for all three. The following is not valid: - #if cond: foo - #elif cond2 - bar - #else: blarg - - - added support for $!silentModePlaceholders - This is the same as quiet mode in Velocity: - http://jakarta.apache.org/velocity/docs/user-guide.html#Quiet%20Reference%20Notation - - - added support for function/method @decorators. It also works with blocks. - As in vanilla Python, the @decorator statement must be followed with a - function/method definition (i.e. #def or #block). - - #from xxx import aDecorator - ... - ... - #@aDecorator - #def func - foo - #end def - - #@aDecorator - #def singleLineShortFormfunc: foo - - #@aDecorator - #block func2 - bar - #end block - - - added a new callback hook 'handlerForExtendsDirective' to the compiler settings. It - can be used to customize the handling of #extends directives. The - callback can dynamically add import statements or rewrite the baseclass' - name if needed: - baseClassName = handler(compiler, baseClassName) - See the discussion on the mailing list on Jan 25th for more details. - - - changed the default filter to the one that doesn't try to encode Unicode - It was 'EncodeUnicode' and is now 'RawOrEncodedUnicode'. - - - added optional support for parsing whitespace between the directive start - token (#) and directive names, per Christophe Eymard's request. For the - argument behind this see the mailing list archives for Jan 29th. This is - off by default. You must turn it on using the compiler setting - allowWhitespaceAfterDirectiveStartToken=True - - #for $something in $another - # for $somethin2 in $another2 - blahblah $something in $something2 - # end for - #end for - - - made the handling of Template.compile()'s preprocessors arg simpler and - fixed a bug in it. - - - fixed attribute name bug in the .compile() method (it affected the feature - that allows generated module files to be cached for better exception - tracebacks) - - - refactored the #cache/CacheRegions code to support abitrary backend cache - data stores. - - - added MemcachedCacheStore, which allows cache data to be stored in a - memcached backend. See http://www.linuxjournal.com/article/7451 and - http://www.danga.com/memcached/. This is only appropriate for systems - running many Python server processes that need to share cached data to - reduce memory requirements. Don't bother with this unless you actually - need it. If you have a limited number of Python server processes it is - much faster, simpler, and more secure to just cache in the memory of each - process. - - KEEP MEMCACHED'S LIMITED SECURITY IN MIND!! It has no authentication or - encryption and will introduce a gaping hole in your defenses unless you - are careful. If you are caching sensitive data you should take measures - to ensure that a) untrusted local system users cannot connect to memcached - server, b) untrusted external servers cannot connect, and c) untrusted - users on trusted external servers cannot connect. Case (a) can be dealt - with via iptable's owner match module for one way to do this: "iptables -A - ... -m owner ..." Cases (b) and (c) can be handled by tunnelling - memcached network connections over stunnel and implementing stunnel - authentication with mandatory peer/client certs. - - - some under-the-hood refactoring of the parser - - - made it possible to add custom directives, or customize the - parsing/handling of existing ones, via the compiler settings - 'directiveNamesAndParsers' and 'endDirectiveNamesAndHandlers' - - - added a compile-time macro facility to Cheetah. These macros are very - similar to macros in Lisp: - http://www.apl.jhu.edu/~hall/Lisp-Notes/Macros.html. - - As with Lisp macros, they take source code (Cheetah source) as input and - return source code (again Cheetah source) as output. They are executed at - compile-time, just like in Lisp and C. The resultant code - gets executed at run-time. - - The new #defmacro directive allows users to create macros inside the - source of their templates. Macros can also be provided via the compiler - setting 'macroDirectives'. The 'macroDirectives' setting allows you to - share common macros between templates. - - The syntax for the opening tag of #defmacro is the same as for #def and - #block. It expects a macro name followed by an optional argument list in - brackets. A `src` argument is automatically added to the beginning of - every macro's argument list. The value of the `src` is the block of - input source code that is provided during a macro call (see below). - - #defmacro [(argspec)] - - #end defmacro - - All of Cheetah's syntax is available for use inside macros, but the - placeholderStartToken is @ instead of $ and the - directiveStartToken/EndToken is % instead of #. Any syntax using the - standard $/# tokens will be treated as plain text and included in the output - of the macro. - - Here are some examples: - #defmacro addHeaderFooter - header - @src - footer - #end defmacro - - #defmacro addHeaderFooter(header='h', footer='f') - @header - @src - @footer - #end defmacro - - There is a single-line short form like for other directives: - - #defmacro addHeaderFooter: header @src footer - #defmacro addHeaderFooter(header='h', footer='f'): @header @src @footer - - The syntax for calling a macro is similar to the simplest usage of the - #call directive: - - #addHeaderFooter - Source $code to wrap - #end addHeaderFooter - - #addHeaderFooter: Source $code to wrap - - #addHeaderFooter header='header', footer='footer: Source $code to wrap - - - In Elisp you write - (defmacro inc (var) - (list 'setq var (list '1+ var))) - to define the macro `inc` and write - (inc x) - which expands to - (setq x (1+ x)) - - In Cheetah you'd write - #defmacro inc: #set @src +=1 - #inc: $i - which expands to - #set $i += 1 - - print Template("""\ - #defmacro inc: #set @src +=1 - #set i = 1 - #inc: $i - $i""").strip()==2 - - - fixed some bugs related to advanced usage of Template.compile(). These - were found via new unit tests. No one had actually run into them yet. - - - added the initial bits of an #i18n directive. It has the same semantics - as - #call self.handleI18n - Some $var cheetah source - #end call - but has a simpler syntax: - #i18n - Some $var cheetah source - #end i18n - - ## single-line short form: - #i18n: Some $var cheetah source - - The method it calls, self.handleI18n, is just a stub at the moment, but it - will soon be a wrapper around gettext. It currently has one required - positional argument `message`. I anticipate supporting the following - optional arguments: - - id = msgid in the translation catalog - domain = translation domain - source = source lang - target = a specific target lang - comment = a comment to the translation team - - plural = the plural form of the message - n = a sized argument to distinguish between single and plural forms - - #i18n is executed at runtime, but it can also be used in conjunction with - a Cheetah preprocessor or macro (see above) to support compile time - translation of strings that don't have to deal with plural forms. - - - added Cheetah.Utils.htmlEncode and Cheetah.Utils.htmlDecode - - - more docstring text - - Unit tests: [TR] - - extended the caching tests - - added tests for the various calling styles of Template.compile() - - added copies of all the SyntaxAndOutput tests that use a template - baseclass other than `Template`. This ensures that all syntax & core - features work with 2.0's support for arbitrary baseclasses. - - added tests for all the new directives and the new single-line short forms - - 2.0rc2 (Jan 13th, 2006) - !!!THIS RELEASE REQUIRES RECOMPILATION OF ALL COMPILED CHEETAH TEMPLATES!!! - Core Changes: [TR] - - fixed some python 2.4isms that slipped in. All the tests pass with Python - 2.2 now - - added lots more docstring content in the Template class - - made multiline comments gobble whitespace like other directives, per JJ's - request. The rather longwinded compiler setting - gobbleWhitespaceAroundMultiLineComments can be used to go back to the old - non-gobbling behaviour if needed. - - added #capture directive to complement the #call directive. - #call executes a region of Cheetah code and passes its output into a function call - #capture executes a region of Cheetah code and assigns its output to a variable - - extended the compile caching code in Template.compile so it works with the - 'file' arg. - - added cacheModuleFilesForTracebacks and cacheDirForModuleFiles args to - Template.compile(). See the docstring for details. - - misc internal refactoring in the parser - - improved handling of keyword args in the __init__ method and fixed a - potential clash between the namespaces and searchList args - - WWW: [TR] - - added the source for the new Cheetah website layout/content - - 2.0rc1 (Jan 10, 2006) - !!!THIS RELEASE REQUIRES RECOMPILATION OF ALL COMPILED CHEETAH TEMPLATES!!! - Core Changes: [TR] - - made it possible nest #filter directives - - added lots more docstring content in the Template class - - added Template.subclass() classmethod for quickly creating subclasses of - existing Cheetah template classes. It takes the same args as the - .compile() classmethod and returns a template that is a subclass of the - template .subclass() is called from: - T1 = Template.compile(' foo - $meth1 - bar\n#def meth1: this is T1.meth1') - T2 = T1.subclass('#implements meth1\n this is T2.meth1') - - - added baseclass arg to Template.compile(). It simplifies the reuse of - dynamically compiled templates: - # example 1, quickly subclassing a normal Python class and using its - # __init__ call signature: - dictTemplate = Template.compile('hello $name from $caller', baseclass=dict) - print dictTemplate(name='world', caller='me') - - # example 2, mixing a Cheetah method into a class definition: - class Foo(dict): - def meth1(self): - return 'foo' - def meth2(self): - return 'bar' - Foo = Template.compile('#implements meth3\nhello $name from $caller', - baseclass=Foo) - print Foo(name='world', caller='me') - - A side-benefit is the possibility to use the same Cheetah source with - several baseclass, as the baseclass is orthogonal to the source code, - unlike the #extends directive. - - - added 'namespaces' as an alias for 'searchList' in Template.__init__ - - made it possible to pass in a single namespace to 'searchList', which will - automatically be converted into a list. - - fixed issue with buffering and use of #call when template is used as a - webkit servlet - - added Cheetah.Utils.htmlEncode and htmlDecode - - The command line tool (CheetahWrapper.py): - - changed insertion order for the --env and --pickle options so they match the - commandline UI of the compiled template modules themselves [TR] - - 2.0b5 (Jan 7, 2006) - !!!THIS RELEASE REQUIRES RECOMPILATION OF ALL COMPILED CHEETAH TEMPLATES!!! - Core Changes: [TR] - - made Cheetah.Template a new-style class by inserting 'object' into its' - inheritance tree. Templates can now use super(), properties and all the - other goodies that come with new-style classes. - - removed the WebInputMixin by placing its one method directly in the - Template class. - - removed the SettingsManager Mixin. It wasn't being used by anything - anymore. - - added a framework for caching the results of compilations in - Template.compile(). This is on by default and protects against bad - performance issues that are due to programmers misguidedly compiling - templates inside tight loops. It also saves on memory usage. - - misc attr name changes to avoid namespace pollution - - more + improved docstrings - - replaced the oldstyle dynamic compile hacks with a wrapper around - Template.compile(). The old usage pattern Template(src) now benefits from - most of the recent changes. - Template(src).__class__ == Template.compile(src) - - removed all the extra imports required by oldstyle dynamic compile hacks - - converted the cheetah #include mechanism to newstyle compilation and made it - more flexible - - made the #include mechanism work with file objects in addition to file names - - made the handling of args to Template.compile() more flexible. You can now - provide defaults via class attributes. - - made preprocessors for Template.compile() work with file arguments - - added support for specifying a __metaclass__ on cheetah template classes - - refactored both the class and instance initialization processes - - improved the handling of __str__ in _assignRequiredMethodsToClass - - The command line tool (CheetahWrapper.py): [TR] - - improved error output in CheetahWrapper - - switched fill command over to new style compile usage - - Unit tests: [TR] - - fixed format string bug in unittest_local_copy.py - - 2.0b4 (Jan 6, 2006) - !!!THIS RELEASE REQUIRES RECOMPILATION OF ALL COMPILED CHEETAH TEMPLATES!!! - Core Changes: [TR] - - fixed up parsing of target lists in for loops. This was previously limited - to fairly simple target lists. - #for ($i, $j) in [('aa','bb'),('cc','dd')] - $i.upper,$j.upper - #end for" - #for (i, j) in [('aa','bb'),('cc','dd')] - $i.upper,$j.upper - #end for" - #for i,(j, k) in enumerate([('aa','bb'),('cc','dd')]) - $j.upper,$k.upper - #end for" - - refactored the class initialization process - - improved handling of target lists in #set directive. This was previously - limited to fairly simple target lists. - #set i,j = [1,2] ... #set $i,$j = [1,2] - #set (i,j) = [1,2] ... #set ($i,$j) = [1,2] - #set i, (j,k) = [1,(2,3)] ... #set $i, ($j,$k) = [1,(2,3)] - - - made it possible for the expressionFilter hooks to modify the code chunks - they are fed. Also documented the hooks in a docstring. Thus the hooks - can be used as preprocessors for expressions, 'restricted execution', or - even enforcement of style guidelines. - - - removed cheetah junk from docstrings and placed it all in comments or - __moduleVars__. Per JJ's suggestion. - - - made it possible to nest #cache directives to any level - - made it possible to nest #call directives to any level - - Unit Tests [TR] - - extended tests for #for directive - - expanded tests for #set directive - - expanded tests for #call directive - - expanded tests for #cache directive - - added basic tests for the new $placeholder string expressions: - c'text $placeholder text' - - 2.0b3 (Jan 5, 2006) - !!!THIS RELEASE REQUIRES RECOMPILATION OF ALL COMPILED CHEETAH TEMPLATES!!! - Core Changes: [TR] - - added #yield statement - - added ability to create nested scopes/functions via nested #def statements - - added new #call directive and related #arg directive, per Ian Bicking's - suggestion. - - added new expression syntax c"text $placeholder text" - - for those basic function calling cases where you just need to pass in a - small bit of cheetah output as an argument: - - c'a string with $placeholders', - c'''a string with $placeholders''', - c"a string with $placeholders", - c"""a string with $placeholders""" - - - They can't contain #directives, but accept any valid $placeholder syntax - except caching placeholders. Caching placeholders don't make any sense in - this context. - - They can be used *any* place where a python expression is expected. - - They can be nested to any depth. - - $func(c'
  • $var1-$var2
    • ') - $func(c'
  • $var1-$var2
    • ', doSomething=True) - $func(content=c'
  • $var1-$var2
    • ', doSomething=True) - $func(lambda x,y: c'
  • $x-$y
    • ') - $func(callback=lambda x,y: c'
  • $x-$y
    • ') - $func(lambda x,y: c'
  • $x-$y-$varInSearchList
    • ') - $func(c'
  • $var1-$var2-$(var3*10)-$(94.3*58)
    • ') - $func(c'
  • $var1-$var2-$func2(c"a nested expr $var99")
    • ') - #if $cond then c'
  • $var1-$var2
    • ' else c'

    $var1-$var2

    ' - #def foo(arg1=c'$var1$var2'): blah $arg1 blah - $foo(c'$var1$var2') - - - added preprocessor hooks to Template.compile() - can be used for partial completion or 'compile-time-caching' - ... more details and examples coming. It's very useful, but takes a bit - of explaining. - - added '#set module varName = expr' for adding module globals. JJ's suggestion - - improved generated docstring notes about cached vars - - fixed silly bug related to """ in docstring comments and statements like - this '#def foo: $str("""foo""")'. Reported by JJ. - - changed the handling of single-line defs so that - '#def xxx:\n' will be treated as a multi-line #def. - The same applies to #block. There's a compiler setting to turn this off - if you really need empty single-line #def:'s. - JJ reported that this was causing great confusion with beginners. - - improved error message for unclosed directives, per Mike Orr's suggestion. - - added optional support for passing the trans arg to methods via **KWS rather - than trans=None. See the discussion on the mailing list Jan 4th (JJ's post) for - details. The purpose is to avoid a positional argument clash that - apparently is very confusing for beginners. - - Note that any existing client code that passing the trans arg in - positionally rather than as a keyword will break as a result. WebKit - does this with the .respond method so I've kept the old style there. - You can also turn this new behaviour off by either manually including - the trans arg in your method signature (see the example below) or by - using the compiler setting 'useKWsDictArgForPassingTrans'=False. - - #def manualOverride(arg1, trans=None) - foo $arg1 - #end def - - ImportHooks: - - made the ImportHook more robust against compilation errors during import [TR] - - Install scripts: [TR] - - added optional support for pje's setuptools - - added cheeseshop classifiers - - removed out of date install instructions in __init__.py - - Servlet Base Class For Webkit: [TR] - - disabled assignment of self.application (was a webware hack) - - Unit Tests: [TR] - - unit tests for most of the new syntax elements - - tidied up some old tests - - misc refactoring - - 2.0b2 (Dec 30, 2005) - !!!THIS RELEASE REQUIRES RECOMPILATION OF ALL COMPILED CHEETAH TEMPLATES!!! - - Core Changes: - - In previous versions of Cheetah tracebacks from exceptions that were raised - inside dynamically compiled Cheetah templates were opaque because - Python didn't have access to a python source file to use in the traceback: - - File "xxxx.py", line 192, in getTextiledContent - content = str(template(searchList=searchList)) - File "cheetah_yyyy.py", line 202, in __str__ - File "cheetah_yyyy.py", line 187, in respond - File "cheetah_yyyy.py", line 139, in writeBody - ZeroDivisionError: integer division or modulo by zero - - It is now possible to keep the generated source code from the python - classes returned by Template.compile() in a cache dir. Having these files - around allows Python to include the actual source lines in tracebacks and - makes them much easier to understand: - - File "/usr/local/unsnarl/lib/python/us/ui/views/WikiPageRenderer.py", line 192, in getTextiledContent - content = str(template(searchList=searchList)) - File "/tmp/CheetahCacheDir/cheetah_yyyy.py", line 202, in __str__ - def __str__(self): return self.respond() - File "/tmp/CheetahCacheDir/cheetah_yyyy.py", line 187, in respond - self.writeBody(trans=trans) - File "/tmp/CheetahCacheDir/cheetah_yyyy.py", line 139, in writeBody - __v = 0/0 # $(0/0) - ZeroDivisionError: integer division or modulo by zero - - This is turned off by default. To turn it on, do this: - - class NiceTracebackTemplate(Template): - _CHEETAH_cacheModuleFilesForTracebacks = True - _CHEETAH_cacheDirForModuleFiles = '/tmp/CheetahCacheDir' # change to a dirname - - templateClass = NiceTracebackTemplate.compile(src) - - # or - templateClass = Template.compile(src, - cacheModuleFilesForTracebacks=True, cacheDirForModuleFiles='/tmp/CheetahCacheDir') - - - This only works with the new Template.compile(src) usage style! - - Note, Cheetah generated modules that are compiled on the command line have - never been affected by this issue. [TR] - - - added an extra comment per $placeholder to generated python code so it is - easier to grok. [TR] - - 2.0b1 (Dec 29, 2005) - !!!THIS RELEASE REQUIRES RECOMPILATION OF ALL COMPILED CHEETAH TEMPLATES!!! - - Core Changes: - - enabled use of any expression in ${placeholders}. See the examples I posted to - the email list on Dec 12th. All use cases of the #echo directive can now - be handled with ${placeholders}. This came from a suggestion by Mike - Orr. [TR] - - - made it possible for templates to #extend (aka subclass) any arbitrary - baseclass, including Python's new style classes. You must either compile - your classes on the command line or use the new classmethod - Template.compile() as described below. The old Template(src) interface - still works, provided you don't try to use this new arbitrary baseclass - stuff. See my messages to the email list for more details. [TR] - - - made it possible to create template classes dynamically, rather than just - instances. See the new classmethod Template.compile(). See my messages - to the email list for more details. [TR] - - klass = Template.compile(src) - - - made it easier to work with custom compiler settings, particularly from - the command line tool. You can now define a subclass of Template which - will compile your templates using custom compilerSettings, or even a - custom compiler class, without requiring you to manually pass in your - compilerSettings each time or define them in the template src itself via - the #compiler directive. You can make the command line tool use your - subclass by defining the environment variable CHEETAH_TEMPLATE_CLASS. It - should be in the form 'package.module:class'. See my messages - to the email list for more details. [TR] - - - made it possible to pass the searchList in as an argument to #def'ined - methods. This makes all lookup that occur within the scope of that method - use the provided searchList rather than self._searchList. This does not - carry over to other methods called within the top method, unless they - explicitly accept the searchList in their signature AND you pass it to - them when calling them. This behaviour can be turned off with the - corresponding compilerSetting 'allowSearchListAsMethArg' [TR] - - - added hooks for filtering / restricting dangerous stuff in cheetah source - code at compile time. These hooks can be used to enable Cheetah template - authoring by untrusted users. See my messages to the email list for more - details. Note, it filters expressions at parse/compile time, unlike Python's - old rexec module which restricted the Python environment at runtime. [TR] - - # Here are the relevant compiler settings: - # use lower case keys here!! - 'disabledDirectives':[], # list of directive keys, without the start token - 'enabledDirectives':[], # list of directive keys, without the start token - - 'disabledDirectiveHooks':[], # callable(parser, directiveKey), - # called when a disabled directive is found, prior to raising an exception - - 'preparseDirectiveHooks':[], # callable(parser, directiveKey) - 'postparseDirectiveHooks':[], # callable(parser, directiveKey) - - 'preparsePlaceholderHooks':[], # callable(parser) - 'postparsePlaceholderHooks':[], # callable(parser) - - 'expressionFilterHooks':[], - # callable(parser, expr, exprType, rawExpr=None, startPos=None) - # exprType is the name of the directive, 'psp', or 'placeholder'. - #all lowercase - - - added support for a short EOLSlurpToken to supplement the #slurp - directive. It's currently re.compile('#\s*\n') (i.e # followed by - arbitrary whitespace and a new line), but this is not set in stone. One - other suggestion was the backslash char, but I believe Python's own - interpretation of backslashes will lead to confusion. The compiler - setting 'EOLSlurpToken' controls this. You can turn it off completely by - setting 'EOLSlurpToken' to None. See the email list for more details. [TR] - - - added '_CHEETAH_' prefix to all instance attribute names in compiled - templates. This is related to the arbitrary baseclass change. [TR] - - - shifted instance attribute setup to _initCheetahAttributes() method. This - is related to the arbitrary baseclass change. [TR] - - - made it possible to use full expressions in the #extends directive, rather - than just dotted names. This allows you to do things like this: - - #from xx.TemplateRepository import getTemplateClass - #extends getTemplateClass('someName') - - I don't expect this to be used much. I needed it for a wiki system in - which the baseclasses for the templates are dynamically compiled at run - time and are not available via simple imports. [TR] - - - added compiler setting autoImportForExtendDirective=True, so this existing - default behaviour can be turned off when needed. [TR] - - - fixed a bug in the parsing of single-line #def's and #block's when they - are enclosed within #if ... #end if. Reported by Marcin Gajda [TR] - - - tweak to remove needless write('') calls in generated code [TR] - - The command line tool (CheetahWrapper.py): - - added code to cleanup trailing slashes on path arguments (code originally - from Mike Orr) [TR] - - turned on the ImportHooks by default for the 'cheetah fill' command. See the - discussion on the email list [TR] - - ImportHooks: - - fixed a name error bug in the ImportHooks [TR] + See http://www.cheetahtemplate.org/CHANGES.txt for full details + Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable === removed file 'README' --- README 2006-07-26 22:03:15 +0000 +++ README 1970-01-01 00:00:00 +0000 @@ -1,51 +0,0 @@ -Cheetah is an open source template engine and code generation tool. - -It can be used standalone or combined with other tools and frameworks. Web -development is its principle use, but Cheetah is very flexible and is also being -used to generate C++ game code, Java, sql, form emails and even Python code. - -Documentation -================================================================================ -For a high-level introduction to Cheetah please refer to the User\'s Guide -at http://cheetahtemplate.org/learn.html - -Mailing list -================================================================================ -cheetahtemplate-discuss@lists.sourceforge.net -Subscribe at http://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss - -Credits -================================================================================ -http://cheetahtemplate.org/credits.html - -Praise -================================================================================ -"I\'m enamored with Cheetah" - Sam Ruby, senior member of IBM Emerging -Technologies Group & director of Apache Software Foundation - -"Give Cheetah a try. You won\'t regret it. ... Cheetah is a truly powerful -system. ... Cheetah is a serious contender for the 'best of breed' Python -templating." - Alex Martelli - -"People with a strong PHP background absolutely love Cheetah for being Smarty, -but much, much better." - Marek Baczynski - -"I am using Smarty and I know it very well, but compiled Cheetah Templates with -its inheritance approach is much powerful and easier to use than Smarty." - -Jaroslaw Zabiello - -"There is no better solution than Cheetah" - Wilk - -"A cheetah template can inherit from a python class, or a cheetah template, and -a Python class can inherit from a cheetah template. This brings the full power -of OO programming facilities to the templating system, and simply blows away -other templating systems" - Mike Meyer - -"Cheetah has successfully been introduced as a replacement for the overweight -XSL Templates for code generation. Despite the power of XSL (and notably XPath -expressions), code generation is better suited to Cheetah as templates are much -easier to implement and manage." - The FEAR development team - (http://fear.sourceforge.net/docs/latest/guide/Build.html#id2550573) - -"I\'ve used Cheetah quite a bit and it\'s a very good package" - Kevin Dangoor, -lead developer of TurboGears. === added file 'README.markdown' --- README.markdown 1970-01-01 00:00:00 +0000 +++ README.markdown 2010-10-19 19:11:10 +0000 @@ -0,0 +1,51 @@ +Cheetah is an open source template engine and code generation tool. + +It can be used standalone or combined with other tools and frameworks. Web +development is its principle use, but Cheetah is very flexible and is also being +used to generate C++ game code, Java, sql, form emails and even Python code. + +Documentation +================================================================================ +For a high-level introduction to Cheetah please refer to the User\'s Guide +at http://cheetahtemplate.org/learn.html + +Mailing list +================================================================================ +cheetahtemplate-discuss@lists.sourceforge.net +Subscribe at http://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss + +Credits +================================================================================ +http://cheetahtemplate.org/credits.html + +Praise +================================================================================ +"I\'m enamored with Cheetah" - Sam Ruby, senior member of IBM Emerging +Technologies Group & director of Apache Software Foundation + +"Give Cheetah a try. You won\'t regret it. ... Cheetah is a truly powerful +system. ... Cheetah is a serious contender for the 'best of breed' Python +templating." - Alex Martelli + +"People with a strong PHP background absolutely love Cheetah for being Smarty, +but much, much better." - Marek Baczynski + +"I am using Smarty and I know it very well, but compiled Cheetah Templates with +its inheritance approach is much powerful and easier to use than Smarty." - +Jaroslaw Zabiello + +"There is no better solution than Cheetah" - Wilk + +"A cheetah template can inherit from a python class, or a cheetah template, and +a Python class can inherit from a cheetah template. This brings the full power +of OO programming facilities to the templating system, and simply blows away +other templating systems" - Mike Meyer + +"Cheetah has successfully been introduced as a replacement for the overweight +XSL Templates for code generation. Despite the power of XSL (and notably XPath +expressions), code generation is better suited to Cheetah as templates are much +easier to implement and manage." - The FEAR development team + (http://fear.sourceforge.net/docs/latest/guide/Build.html#id2550573) + +"I\'ve used Cheetah quite a bit and it\'s a very good package" - Kevin Dangoor, +lead developer of TurboGears. === modified file 'SetupConfig.py' --- SetupConfig.py 2006-07-26 22:03:15 +0000 +++ SetupConfig.py 2010-10-19 19:11:10 +0000 @@ -1,17 +1,18 @@ #-------Main Package Settings-----------# -name = "Cheetah" -from src.Version import Version as version -maintainer = "Tavis Rudd" +import sys + +name = 'Cheetah' +from cheetah.Version import Version as version +maintainer = "R. Tyler Ballance" author = "Tavis Rudd" author_email = "cheetahtemplate-discuss@lists.sf.net" -url = "http://www.CheetahTemplate.org/" +url = "http://www.cheetahtemplate.org/" packages = ['Cheetah', 'Cheetah.Macros', 'Cheetah.Templates', 'Cheetah.Tests', 'Cheetah.Tools', 'Cheetah.Utils', - 'Cheetah.Utils.optik', ] classifiers = [line.strip() for line in '''\ #Development Status :: 4 - Beta @@ -28,30 +29,50 @@ Topic :: Software Development :: Libraries :: Python Modules Topic :: Software Development :: User Interfaces Topic :: Text Processing'''.splitlines() if not line.strip().startswith('#')] -del line -package_dir = {'Cheetah':'src'} +package_dir = {'Cheetah':'cheetah'} import os import os.path from distutils.core import Extension -## we only assume the presence of a c compiler on Posix systems, NT people will -# have to enable this manually. -if os.name == 'posix': - ext_modules=[Extension("Cheetah._namemapper", [os.path.join("src" ,"_namemapper.c")] - ) - ] -else: - ext_modules=[] - +ext_modules=[ + Extension("Cheetah._namemapper", + [os.path.join('cheetah', 'c', '_namemapper.c')]), + # Extension("Cheetah._verifytype", + # [os.path.join('cheetah', 'c', '_verifytype.c')]), + # Extension("Cheetah._filters", + # [os.path.join('cheetah', 'c', '_filters.c')]), + # Extension('Cheetah._template', + # [os.path.join('cheetah', 'c', '_template.c')]), + ] ## Data Files and Scripts -scripts = ['bin/cheetah-compile', +scripts = ('bin/cheetah-compile', 'bin/cheetah', - ] -data_files = ['recursive: src *.tmpl *.txt LICENSE README TODO CHANGES', - ] + 'bin/cheetah-analyze', + ) + +data_files = ['recursive: cheetah *.tmpl *.txt LICENSE README TODO CHANGES',] + +if not os.getenv('CHEETAH_INSTALL_WITHOUT_SETUPTOOLS'): + try: + from setuptools import setup + install_requires = [ + "Markdown >= 2.0.1", + ] + if sys.platform == 'win32': + # use 'entry_points' instead of 'scripts' + del scripts + entry_points = { + 'console_scripts': [ + 'cheetah = Cheetah.CheetahWrapper:_cheetah', + 'cheetah-compile = Cheetah.CheetahWrapper:_cheetah_compile', + ] + } + except ImportError: + print('Not using setuptools, so we cannot install the Markdown dependency') + description = "Cheetah is a template engine and code generation tool." @@ -64,7 +85,7 @@ Documentation ================================================================================ For a high-level introduction to Cheetah please refer to the User\'s Guide -at http://cheetahtemplate.org/learn.html +at http://www.cheetahtemplate.org/learn.html Mailing list ================================================================================ @@ -73,48 +94,10 @@ Credits ================================================================================ -http://cheetahtemplate.org/credits.html - -Praise -================================================================================ -"I\'m enamored with Cheetah" - Sam Ruby, senior member of IBM Emerging -Technologies Group & director of Apache Software Foundation - -"Give Cheetah a try. You won\'t regret it. ... Cheetah is a truly powerful -system. ... Cheetah is a serious contender for the 'best of breed' Python -templating." - Alex Martelli - -"People with a strong PHP background absolutely love Cheetah for being Smarty, -but much, much better." - Marek Baczynski - -"I am using Smarty and I know it very well, but compiled Cheetah Templates with -its inheritance approach is much powerful and easier to use than Smarty." - -Jaroslaw Zabiello - -"There is no better solution than Cheetah" - Wilk - -"A cheetah template can inherit from a python class, or a cheetah template, and -a Python class can inherit from a cheetah template. This brings the full power -of OO programming facilities to the templating system, and simply blows away -other templating systems" - Mike Meyer - -"Cheetah has successfully been introduced as a replacement for the overweight -XSL Templates for code generation. Despite the power of XSL (and notably XPath -expressions), code generation is better suited to Cheetah as templates are much -easier to implement and manage." - The FEAR development team - (http://fear.sourceforge.net/docs/latest/guide/Build.html#id2550573) - -"I\'ve used Cheetah quite a bit and it\'s a very good package" - Kevin Dangoor, -lead developer of TurboGears. +http://www.cheetahtemplate.org/credits.html Recent Changes ================================================================================ -See http://cheetahtemplate.org/docs/CHANGES for full details. +See http://www.cheetahtemplate.org/CHANGES.txt for full details ''' -try: - recentChanges = open('CHANGES').read().split('\n1.0')[0] - long_description += recentChanges - del recentChanges -except: - pass === modified file 'SetupTools.py' --- SetupTools.py 2007-11-18 14:59:06 +0000 +++ SetupTools.py 2010-10-19 19:11:10 +0000 @@ -1,44 +1,61 @@ #!/usr/bin/env python -# $Id: SetupTools.py,v 1.9 2007/11/03 19:44:38 tavis_rudd Exp $ -"""Some tools for extending and working with distutils - -CREDITS: This module borrows code and ideas from M.A. Lemburg's excellent setup -tools for the mxBase package. - -""" - -__author__ = "Tavis Rudd " -__version__ = "$Revision: 1.9 $"[11:-2] - import os from os import listdir import os.path from os.path import exists, isdir, isfile, join, splitext +import sys import types import glob import string import traceback from distutils.core import setup -if 'CHEETAH_USE_SETUPTOOLS' in os.environ: - # @@TR: Please note that this is for testing purposes only! PEAK setuptools - # is not required or recommended for installing Cheetah. Downstream - # package managers (linux distros, etc.) should *not* enable this. +if not os.getenv('CHEETAH_INSTALL_WITHOUT_SETUPTOOLS'): try: - # use http://peak.telecommunity.com/DevCenter/setuptools if it's installed - # requires Py >=2.3 from setuptools import setup except ImportError: from distutils.core import setup from distutils.core import Command +from distutils.command.build_ext import build_ext from distutils.command.install_data import install_data +from distutils.errors import CCompilerError, DistutilsExecError, \ + DistutilsPlatformError #imports from Cheetah ... -from src.FileUtils import findFiles +from cheetah.FileUtils import findFiles + +if sys.platform == 'win32' and sys.version_info > (2, 6): + # 2.6's distutils.msvc9compiler can raise an IOError when failing to + # find the compiler + ext_errors = (CCompilerError, DistutilsExecError, DistutilsPlatformError, + IOError) +else: + ext_errors = (CCompilerError, DistutilsExecError, DistutilsPlatformError) ################################################## ## CLASSES ## + +class BuildFailed(Exception): + pass + +class mod_build_ext(build_ext): + """A modified version of the distutils build_ext command that raises an + exception when building of the extension fails. + """ + + def run(self): + try: + build_ext.run(self) + except DistutilsPlatformError, x: + raise BuildFailed(x) + + def build_extension(self, ext): + try: + build_ext.build_extension(self, ext) + except ext_errors, x: + raise BuildFailed(x) + class mod_install_data(install_data): """A modified version of the disutils install_data command that allows data @@ -60,8 +77,8 @@ data_files = self.get_inputs() for entry in data_files: - if type(entry) != types.StringType: - raise ValueError, 'The entries in "data_files" must be strings' + if not isinstance(entry, basestring): + raise ValueError('The entries in "data_files" must be strings') entry = string.join(string.split(entry, '/'), os.sep) # entry is a filename or glob pattern @@ -117,25 +134,39 @@ """ # Build parameter dictionary kws = {} + newkws = {} for configuration in configurations: kws.update(vars(configuration)) for name, value in kws.items(): - if name[:1] == '_' or \ - type(value) not in (types.StringType, - types.ListType, - types.TupleType, - types.DictType, - types.IntType, - ): - del kws[name] + if name[:1] == '_': + continue + if not isinstance(value, (basestring, list, tuple, dict, int)): + continue + newkws[name] = value + kws = newkws # Add setup extensions cmdclasses = { + 'build_ext': mod_build_ext, 'install_data': mod_install_data, } kws['cmdclass'] = cmdclasses # Invoke distutils setup - apply(setup, (), kws) + try: + setup(**kws) + except BuildFailed, x: + print("One or more C extensions failed to build.") + print("Details: %s" % x) + if os.environ.get('CHEETAH_C_EXTENSIONS_REQUIRED'): + raise x + print("Retrying without C extensions enabled.") + + del kws['ext_modules'] + setup(**kws) + + print("One or more C extensions failed to build.") + print("Performance enhancements will not be available.") + print("Pure Python installation succeeded.") === modified file 'TODO' --- TODO 2007-07-11 14:45:50 +0000 +++ TODO 2010-10-19 19:11:10 +0000 @@ -1,16 +1,9 @@ -Cheetah TODO list ------------------ -* If you are working on a task please put your initials at the end of the - description -* When a task is completed please remember to note it in the CHANGES file -* Unresolved bugs are listed in the BUGS file. Resolved bugs are be listed - in the CHANGES file if the bug is considered significant enough and it - affected a released version of Cheetah. - -Required for Cheetah 2.0 -======================== -- Replace Optik with Python's optparse. Optik license has been removed from - Users' Guide. +NOTE: Please see http://bugs.cheetahtemplate.org + for future feature requests/bugs/TODO + + +=============================================================================== +=============================================================================== Desired for Cheetah 2.0 ======================= @@ -38,7 +31,6 @@ leak from one fill to the next. - CheetahWrapper stuff: (MO) - * "cheetah compile --shbang '#!/usr/bin/python2.2'" * "cheetah preview [options] [FILES]" print template-specific portion of main method(s) to stdout, with line numbers based on the .py template module. Make a Template method to do the same thing, a la .generatedModuleCode(). === modified file 'bin/cheetah' --- bin/cheetah 2004-03-30 18:47:41 +0000 +++ bin/cheetah 2010-10-19 19:11:10 +0000 @@ -1,3 +1,3 @@ #!/usr/bin/env python -from Cheetah.CheetahWrapper import CheetahWrapper -CheetahWrapper().main() +from Cheetah.CheetahWrapper import _cheetah +_cheetah() === added file 'bin/cheetah-analyze' --- bin/cheetah-analyze 1970-01-01 00:00:00 +0000 +++ bin/cheetah-analyze 2010-10-19 19:11:10 +0000 @@ -0,0 +1,6 @@ +#!/usr/bin/env python + +from Cheetah import DirectiveAnalyzer + +if __name__ == '__main__': + DirectiveAnalyzer.main() === modified file 'bin/cheetah-compile' --- bin/cheetah-compile 2004-03-30 18:47:41 +0000 +++ bin/cheetah-compile 2010-10-19 19:11:10 +0000 @@ -1,5 +1,3 @@ #!/usr/bin/env python -import sys -from Cheetah.CheetahWrapper import CheetahWrapper -sys.argv.insert(1, "compile") -CheetahWrapper().main() +from Cheetah.CheetahWrapper import _cheetah_compile +_cheetah_compile() === removed file 'cachedCompile.py' --- cachedCompile.py 2006-07-26 22:03:15 +0000 +++ cachedCompile.py 1970-01-01 00:00:00 +0000 @@ -1,11 +0,0 @@ -from Cheetah.Template import Template -source = file('/home/tavis/cvs_working/Cheetah/src/Templates/SkeletonPage.tmpl').read() -##klass = Template.compile(source, -## cacheCompilationResults=1, -## useCache=1, -## ) -for i in range(2000): - klass = Template.compile(source, - cacheCompilationResults=0, - useCache=0, - ) === added directory 'cheetah' === removed file 'cheetah-mem.py' --- cheetah-mem.py 2006-07-26 22:03:15 +0000 +++ cheetah-mem.py 1970-01-01 00:00:00 +0000 @@ -1,22 +0,0 @@ -from Cheetah.Template import Template -import gc - -src = open('/tmp/z.py').read() -tclass = Template.compile(src) -#t = Template(src) -nr = 0 -while True: - #tclass = Template.compile(src) - #t = tclass() - t = Template(src) - - output = t.respond() - - nr += 1 - if not nr % 10000: - print - #print 'collect' - #gc.collect() - - print 'tclass id', id(t.__class__) - print 'cache size', len(Template._CHEETAH_compileCache.keys()) === added file 'cheetah/CacheRegion.py' --- cheetah/CacheRegion.py 1970-01-01 00:00:00 +0000 +++ cheetah/CacheRegion.py 2010-10-19 19:11:10 +0000 @@ -0,0 +1,136 @@ +# $Id: CacheRegion.py,v 1.3 2006/01/28 04:19:30 tavis_rudd Exp $ +''' +Cache holder classes for Cheetah: + +Cache regions are defined using the #cache Cheetah directive. Each +cache region can be viewed as a dictionary (keyed by cacheRegionID) +handling at least one cache item (the default one). It's possible to add +cacheItems in a region by using the `varyBy` #cache directive parameter as +in the following example:: + #def getArticle + this is the article content. + #end def + + #cache varyBy=$getArticleID() + $getArticle($getArticleID()) + #end cache + +The code above will generate a CacheRegion and add new cacheItem for each value +of $getArticleID(). +''' + +try: + from hashlib import md5 +except ImportError: + from md5 import md5 + +import time +import Cheetah.CacheStore + +class CacheItem(object): + ''' + A CacheItem is a container storing: + + - cacheID (string) + - refreshTime (timestamp or None) : last time the cache was refreshed + - data (string) : the content of the cache + ''' + + def __init__(self, cacheItemID, cacheStore): + self._cacheItemID = cacheItemID + self._cacheStore = cacheStore + self._refreshTime = None + self._expiryTime = 0 + + def hasExpired(self): + return (self._expiryTime and time.time() > self._expiryTime) + + def setExpiryTime(self, time): + self._expiryTime = time + + def getExpiryTime(self): + return self._expiryTime + + def setData(self, data): + self._refreshTime = time.time() + self._cacheStore.set(self._cacheItemID, data, self._expiryTime) + + def getRefreshTime(self): + return self._refreshTime + + def getData(self): + assert self._refreshTime + return self._cacheStore.get(self._cacheItemID) + + def renderOutput(self): + """Can be overridden to implement edge-caching""" + return self.getData() or "" + + def clear(self): + self._cacheStore.delete(self._cacheItemID) + self._refreshTime = None + +class _CacheDataStoreWrapper(object): + def __init__(self, dataStore, keyPrefix): + self._dataStore = dataStore + self._keyPrefix = keyPrefix + + def get(self, key): + return self._dataStore.get(self._keyPrefix+key) + + def delete(self, key): + self._dataStore.delete(self._keyPrefix+key) + + def set(self, key, val, time=0): + self._dataStore.set(self._keyPrefix+key, val, time=time) + +class CacheRegion(object): + ''' + A `CacheRegion` stores some `CacheItem` instances. + + This implementation stores the data in the memory of the current process. + If you need a more advanced data store, create a cacheStore class that works + with Cheetah's CacheStore protocol and provide it as the cacheStore argument + to __init__. For example you could use + Cheetah.CacheStore.MemcachedCacheStore, a wrapper around the Python + memcached API (http://www.danga.com/memcached). + ''' + _cacheItemClass = CacheItem + + def __init__(self, regionID, templateCacheIdPrefix='', cacheStore=None): + self._isNew = True + self._regionID = regionID + self._templateCacheIdPrefix = templateCacheIdPrefix + if not cacheStore: + cacheStore = Cheetah.CacheStore.MemoryCacheStore() + self._cacheStore = cacheStore + self._wrappedCacheDataStore = _CacheDataStoreWrapper( + cacheStore, keyPrefix=templateCacheIdPrefix+':'+regionID+':') + self._cacheItems = {} + + def isNew(self): + return self._isNew + + def clear(self): + " drop all the caches stored in this cache region " + for cacheItemId in self._cacheItems.keys(): + cacheItem = self._cacheItems[cacheItemId] + cacheItem.clear() + del self._cacheItems[cacheItemId] + + def getCacheItem(self, cacheItemID): + """ Lazy access to a cacheItem + + Try to find a cache in the stored caches. If it doesn't + exist, it's created. + + Returns a `CacheItem` instance. + """ + cacheItemID = md5(str(cacheItemID)).hexdigest() + + if cacheItemID not in self._cacheItems: + cacheItem = self._cacheItemClass( + cacheItemID=cacheItemID, cacheStore=self._wrappedCacheDataStore) + self._cacheItems[cacheItemID] = cacheItem + self._isNew = False + return self._cacheItems[cacheItemID] === added file 'cheetah/CacheStore.py' --- cheetah/CacheStore.py 1970-01-01 00:00:00 +0000 +++ cheetah/CacheStore.py 2010-10-19 19:11:10 +0000 @@ -0,0 +1,106 @@ +''' +Provides several CacheStore backends for Cheetah's caching framework. The +methods provided by these classes have the same semantics as those in the +python-memcached API, except for their return values: + +set(key, val, time=0) + set the value unconditionally +add(key, val, time=0) + set only if the server doesn't already have this key +replace(key, val, time=0) + set only if the server already have this key +get(key, val) + returns val or raises a KeyError +delete(key) + deletes or raises a KeyError +''' +import time + +class Error(Exception): + pass + +class AbstractCacheStore(object): + + def set(self, key, val, time=None): + raise NotImplementedError + + def add(self, key, val, time=None): + raise NotImplementedError + + def replace(self, key, val, time=None): + raise NotImplementedError + + def delete(self, key): + raise NotImplementedError + + def get(self, key): + raise NotImplementedError + +class MemoryCacheStore(AbstractCacheStore): + def __init__(self): + self._data = {} + + def set(self, key, val, time=0): + self._data[key] = (val, time) + + def add(self, key, val, time=0): + if key in self._data: + raise Error('a value for key %r is already in the cache'%key) + self._data[key] = (val, time) + + def replace(self, key, val, time=0): + if key in self._data: + raise Error('a value for key %r is already in the cache'%key) + self._data[key] = (val, time) + + def delete(self, key): + del self._data[key] + + def get(self, key): + (val, exptime) = self._data[key] + if exptime and time.time() > exptime: + del self._data[key] + raise KeyError(key) + else: + return val + + def clear(self): + self._data.clear() + +class MemcachedCacheStore(AbstractCacheStore): + servers = ('127.0.0.1:11211') + def __init__(self, servers=None, debug=False): + if servers is None: + servers = self.servers + from memcache import Client as MemcachedClient + self._client = MemcachedClient(servers, debug) + + def set(self, key, val, time=0): + self._client.set(key, val, time) + + def add(self, key, val, time=0): + res = self._client.add(key, val, time) + if not res: + raise Error('a value for key %r is already in the cache'%key) + self._data[key] = (val, time) + + def replace(self, key, val, time=0): + res = self._client.replace(key, val, time) + if not res: + raise Error('a value for key %r is already in the cache'%key) + self._data[key] = (val, time) + + def delete(self, key): + res = self._client.delete(key, time=0) + if not res: + raise KeyError(key) + + def get(self, key): + val = self._client.get(key) + if val is None: + raise KeyError(key) + else: + return val + + def clear(self): + self._client.flush_all() === added file 'cheetah/CheetahWrapper.py' --- cheetah/CheetahWrapper.py 1970-01-01 00:00:00 +0000 +++ cheetah/CheetahWrapper.py 2010-10-19 19:11:10 +0000 @@ -0,0 +1,632 @@ +# $Id: CheetahWrapper.py,v 1.26 2007/10/02 01:22:04 tavis_rudd Exp $ +"""Cheetah command-line interface. + +2002-09-03 MSO: Total rewrite. +2002-09-04 MSO: Bugfix, compile command was using wrong output ext. +2002-11-08 MSO: Another rewrite. + +Meta-Data +================================================================================ +Author: Tavis Rudd and Mike Orr > +Version: $Revision: 1.26 $ +Start Date: 2001/03/30 +Last Revision Date: $Date: 2007/10/02 01:22:04 $ +""" +__author__ = "Tavis Rudd and Mike Orr " +__revision__ = "$Revision: 1.26 $"[11:-2] + +import getopt, glob, os, pprint, re, shutil, sys +import cPickle as pickle +from optparse import OptionParser + +from Cheetah.Version import Version +from Cheetah.Template import Template, DEFAULT_COMPILER_SETTINGS +from Cheetah.Utils.Misc import mkdirsWithPyInitFiles + +optionDashesRE = re.compile( R"^-{1,2}" ) +moduleNameRE = re.compile( R"^[a-zA-Z_][a-zA-Z_0-9]*$" ) + +def fprintfMessage(stream, format, *args): + if format[-1:] == '^': + format = format[:-1] + else: + format += '\n' + if args: + message = format % args + else: + message = format + stream.write(message) + +class Error(Exception): + pass + + +class Bundle: + """Wrap the source, destination and backup paths in one neat little class. + Used by CheetahWrapper.getBundles(). + """ + def __init__(self, **kw): + self.__dict__.update(kw) + + def __repr__(self): + return "" % self.__dict__ + + +################################################## +## USAGE FUNCTION & MESSAGES + +def usage(usageMessage, errorMessage="", out=sys.stderr): + """Write help text, an optional error message, and abort the program. + """ + out.write(WRAPPER_TOP) + out.write(usageMessage) + exitStatus = 0 + if errorMessage: + out.write('\n') + out.write("*** USAGE ERROR ***: %s\n" % errorMessage) + exitStatus = 1 + sys.exit(exitStatus) + + +WRAPPER_TOP = """\ + __ ____________ __ + \ \/ \/ / + \/ * * \/ CHEETAH %(Version)s Command-Line Tool + \ | / + \ ==----== / by Tavis Rudd + \__________/ and Mike Orr + +""" % globals() + + +HELP_PAGE1 = """\ +USAGE: +------ + cheetah compile [options] [FILES ...] : Compile template definitions + cheetah fill [options] [FILES ...] : Fill template definitions + cheetah help : Print this help message + cheetah options : Print options help message + cheetah test [options] : Run Cheetah's regression tests + : (same as for unittest) + cheetah version : Print Cheetah version number + +You may abbreviate the command to the first letter; e.g., 'h' == 'help'. +If FILES is a single "-", read standard input and write standard output. +Run "cheetah options" for the list of valid options. +""" + +################################################## +## CheetahWrapper CLASS + +class CheetahWrapper(object): + MAKE_BACKUPS = True + BACKUP_SUFFIX = ".bak" + _templateClass = None + _compilerSettings = None + + def __init__(self): + self.progName = None + self.command = None + self.opts = None + self.pathArgs = None + self.sourceFiles = [] + self.searchList = [] + self.parser = None + + ################################################## + ## MAIN ROUTINE + + def main(self, argv=None): + """The main program controller.""" + + if argv is None: + argv = sys.argv + + # Step 1: Determine the command and arguments. + try: + self.progName = progName = os.path.basename(argv[0]) + self.command = command = optionDashesRE.sub("", argv[1]) + if command == 'test': + self.testOpts = argv[2:] + else: + self.parseOpts(argv[2:]) + except IndexError: + usage(HELP_PAGE1, "not enough command-line arguments") + + # Step 2: Call the command + meths = (self.compile, self.fill, self.help, self.options, + self.test, self.version) + for meth in meths: + methName = meth.__name__ + # Or meth.im_func.func_name + # Or meth.func_name (Python >= 2.1 only, sometimes works on 2.0) + methInitial = methName[0] + if command in (methName, methInitial): + sys.argv[0] += (" " + methName) + # @@MO: I don't necessarily agree sys.argv[0] should be + # modified. + meth() + return + # If none of the commands matched. + usage(HELP_PAGE1, "unknown command '%s'" % command) + + def parseOpts(self, args): + C, D, W = self.chatter, self.debug, self.warn + self.isCompile = isCompile = self.command[0] == 'c' + defaultOext = isCompile and ".py" or ".html" + self.parser = OptionParser() + pao = self.parser.add_option + pao("--idir", action="store", dest="idir", default='', help='Input directory (defaults to current directory)') + pao("--odir", action="store", dest="odir", default="", help='Output directory (defaults to current directory)') + pao("--iext", action="store", dest="iext", default=".tmpl", help='File input extension (defaults: compile: .tmpl, fill: .tmpl)') + pao("--oext", action="store", dest="oext", default=defaultOext, help='File output extension (defaults: compile: .py, fill: .html)') + pao("-R", action="store_true", dest="recurse", default=False, help='Recurse through subdirectories looking for input files') + pao("--stdout", "-p", action="store_true", dest="stdout", default=False, help='Send output to stdout instead of writing to a file') + pao("--quiet", action="store_false", dest="verbose", default=True, help='Do not print informational messages to stdout') + pao("--debug", action="store_true", dest="debug", default=False, help='Print diagnostic/debug information to stderr') + pao("--env", action="store_true", dest="env", default=False, help='Pass the environment into the search list') + pao("--pickle", action="store", dest="pickle", default="", help='Unpickle FILE and pass it through in the search list') + pao("--flat", action="store_true", dest="flat", default=False, help='Do not build destination subdirectories') + pao("--nobackup", action="store_true", dest="nobackup", default=False, help='Do not make backup files when generating new ones') + pao("--settings", action="store", dest="compilerSettingsString", default=None, help='String of compiler settings to pass through, e.g. --settings="useNameMapper=False,useFilters=False"') + pao('--print-settings', action='store_true', dest='print_settings', help='Print out the list of available compiler settings') + pao("--templateAPIClass", action="store", dest="templateClassName", default=None, help='Name of a subclass of Cheetah.Template.Template to use for compilation, e.g. MyTemplateClass') + pao("--parallel", action="store", type="int", dest="parallel", default=1, help='Compile/fill templates in parallel, e.g. --parallel=4') + pao('--shbang', dest='shbang', default='#!/usr/bin/env python', help='Specify the shbang to place at the top of compiled templates, e.g. --shbang="#!/usr/bin/python2.6"') + + opts, files = self.parser.parse_args(args) + self.opts = opts + if sys.platform == "win32": + new_files = [] + for spec in files: + file_list = glob.glob(spec) + if file_list: + new_files.extend(file_list) + else: + new_files.append(spec) + files = new_files + self.pathArgs = files + + D("""\ +cheetah compile %s +Options are +%s +Files are %s""", args, pprint.pformat(vars(opts)), files) + + + if opts.print_settings: + print() + print('>> Available Cheetah compiler settings:') + from Cheetah.Compiler import _DEFAULT_COMPILER_SETTINGS + listing = _DEFAULT_COMPILER_SETTINGS + listing.sort(key=lambda l: l[0][0].lower()) + + for l in listing: + print('\t%s (default: "%s")\t%s' % l) + sys.exit(0) + + #cleanup trailing path separators + seps = [sep for sep in [os.sep, os.altsep] if sep] + for attr in ['idir', 'odir']: + for sep in seps: + path = getattr(opts, attr, None) + if path and path.endswith(sep): + path = path[:-len(sep)] + setattr(opts, attr, path) + break + + self._fixExts() + if opts.env: + self.searchList.insert(0, os.environ) + if opts.pickle: + f = open(opts.pickle, 'rb') + unpickled = pickle.load(f) + f.close() + self.searchList.insert(0, unpickled) + + ################################################## + ## COMMAND METHODS + + def compile(self): + self._compileOrFill() + + def fill(self): + from Cheetah.ImportHooks import install + install() + self._compileOrFill() + + def help(self): + usage(HELP_PAGE1, "", sys.stdout) + + def options(self): + return self.parser.print_help() + + def test(self): + # @@MO: Ugly kludge. + TEST_WRITE_FILENAME = 'cheetah_test_file_creation_ability.tmp' + try: + f = open(TEST_WRITE_FILENAME, 'w') + except: + sys.exit("""\ +Cannot run the tests because you don't have write permission in the current +directory. The tests need to create temporary files. Change to a directory +you do have write permission to and re-run the tests.""") + else: + f.close() + os.remove(TEST_WRITE_FILENAME) + # @@MO: End ugly kludge. + from Cheetah.Tests import Test + import unittest + verbosity = 1 + if '-q' in self.testOpts: + verbosity = 0 + if '-v' in self.testOpts: + verbosity = 2 + runner = unittest.TextTestRunner(verbosity=verbosity) + runner.run(unittest.TestSuite(Test.suites)) + + def version(self): + print(Version) + + # If you add a command, also add it to the 'meths' variable in main(). + + ################################################## + ## LOGGING METHODS + + def chatter(self, format, *args): + """Print a verbose message to stdout. But don't if .opts.stdout is + true or .opts.verbose is false. + """ + if self.opts.stdout or not self.opts.verbose: + return + fprintfMessage(sys.stdout, format, *args) + + + def debug(self, format, *args): + """Print a debugging message to stderr, but don't if .debug is + false. + """ + if self.opts.debug: + fprintfMessage(sys.stderr, format, *args) + + def warn(self, format, *args): + """Always print a warning message to stderr. + """ + fprintfMessage(sys.stderr, format, *args) + + def error(self, format, *args): + """Always print a warning message to stderr and exit with an error code. + """ + fprintfMessage(sys.stderr, format, *args) + sys.exit(1) + + ################################################## + ## HELPER METHODS + + + def _fixExts(self): + assert self.opts.oext, "oext is empty!" + iext, oext = self.opts.iext, self.opts.oext + if iext and not iext.startswith("."): + self.opts.iext = "." + iext + if oext and not oext.startswith("."): + self.opts.oext = "." + oext + + + + def _compileOrFill(self): + C, D, W = self.chatter, self.debug, self.warn + opts, files = self.opts, self.pathArgs + if files == ["-"]: + self._compileOrFillStdin() + return + elif not files and opts.recurse: + which = opts.idir and "idir" or "current" + C("Drilling down recursively from %s directory.", which) + sourceFiles = [] + dir = os.path.join(self.opts.idir, os.curdir) + os.path.walk(dir, self._expandSourceFilesWalk, sourceFiles) + elif not files: + usage(HELP_PAGE1, "Neither files nor -R specified!") + else: + sourceFiles = self._expandSourceFiles(files, opts.recurse, True) + sourceFiles = [os.path.normpath(x) for x in sourceFiles] + D("All source files found: %s", sourceFiles) + bundles = self._getBundles(sourceFiles) + D("All bundles: %s", pprint.pformat(bundles)) + if self.opts.flat: + self._checkForCollisions(bundles) + + # In parallel mode a new process is forked for each template + # compilation, out of a pool of size self.opts.parallel. This is not + # really optimal in all cases (e.g. probably wasteful for small + # templates), but seems to work well in real life for me. + # + # It also won't work for Windows users, but I'm not going to lose any + # sleep over that. + if self.opts.parallel > 1: + bad_child_exit = 0 + pid_pool = set() + + def child_wait(): + pid, status = os.wait() + pid_pool.remove(pid) + return os.WEXITSTATUS(status) + + while bundles: + b = bundles.pop() + pid = os.fork() + if pid: + pid_pool.add(pid) + else: + self._compileOrFillBundle(b) + sys.exit(0) + + if len(pid_pool) == self.opts.parallel: + bad_child_exit = child_wait() + if bad_child_exit: + break + + while pid_pool: + child_exit = child_wait() + if not bad_child_exit: + bad_child_exit = child_exit + + if bad_child_exit: + sys.exit("Child process failed, exited with code %d" % bad_child_exit) + + else: + for b in bundles: + self._compileOrFillBundle(b) + + def _checkForCollisions(self, bundles): + """Check for multiple source paths writing to the same destination + path. + """ + C, D, W = self.chatter, self.debug, self.warn + isError = False + dstSources = {} + for b in bundles: + if b.dst in dstSources: + dstSources[b.dst].append(b.src) + else: + dstSources[b.dst] = [b.src] + keys = sorted(dstSources.keys()) + for dst in keys: + sources = dstSources[dst] + if len(sources) > 1: + isError = True + sources.sort() + fmt = "Collision: multiple source files %s map to one destination file %s" + W(fmt, sources, dst) + if isError: + what = self.isCompile and "Compilation" or "Filling" + sys.exit("%s aborted due to collisions" % what) + + + def _expandSourceFilesWalk(self, arg, dir, files): + """Recursion extension for .expandSourceFiles(). + This method is a callback for os.path.walk(). + 'arg' is a list to which successful paths will be appended. + """ + iext = self.opts.iext + for f in files: + path = os.path.join(dir, f) + if path.endswith(iext) and os.path.isfile(path): + arg.append(path) + elif os.path.islink(path) and os.path.isdir(path): + os.path.walk(path, self._expandSourceFilesWalk, arg) + # If is directory, do nothing; 'walk' will eventually get it. + + + def _expandSourceFiles(self, files, recurse, addIextIfMissing): + """Calculate source paths from 'files' by applying the + command-line options. + """ + C, D, W = self.chatter, self.debug, self.warn + idir = self.opts.idir + iext = self.opts.iext + files = [] + for f in self.pathArgs: + oldFilesLen = len(files) + D("Expanding %s", f) + path = os.path.join(idir, f) + pathWithExt = path + iext # May or may not be valid. + if os.path.isdir(path): + if recurse: + os.path.walk(path, self._expandSourceFilesWalk, files) + else: + raise Error("source file '%s' is a directory" % path) + elif os.path.isfile(path): + files.append(path) + elif (addIextIfMissing and not path.endswith(iext) and + os.path.isfile(pathWithExt)): + files.append(pathWithExt) + # Do not recurse directories discovered by iext appending. + elif os.path.exists(path): + W("Skipping source file '%s', not a plain file.", path) + else: + W("Skipping source file '%s', not found.", path) + if len(files) > oldFilesLen: + D(" ... found %s", files[oldFilesLen:]) + return files + + + def _getBundles(self, sourceFiles): + flat = self.opts.flat + idir = self.opts.idir + iext = self.opts.iext + nobackup = self.opts.nobackup + odir = self.opts.odir + oext = self.opts.oext + idirSlash = idir + os.sep + bundles = [] + for src in sourceFiles: + # 'base' is the subdirectory plus basename. + base = src + if idir and src.startswith(idirSlash): + base = src[len(idirSlash):] + if iext and base.endswith(iext): + base = base[:-len(iext)] + basename = os.path.basename(base) + if flat: + dst = os.path.join(odir, basename + oext) + else: + dbn = basename + if odir and base.startswith(os.sep): + odd = odir + while odd != '': + idx = base.find(odd) + if idx == 0: + dbn = base[len(odd):] + if dbn[0] == '/': + dbn = dbn[1:] + break + odd = os.path.dirname(odd) + if odd == '/': + break + dst = os.path.join(odir, dbn + oext) + else: + dst = os.path.join(odir, base + oext) + bak = dst + self.BACKUP_SUFFIX + b = Bundle(src=src, dst=dst, bak=bak, base=base, basename=basename) + bundles.append(b) + return bundles + + + def _getTemplateClass(self): + C, D, W = self.chatter, self.debug, self.warn + modname = None + if self._templateClass: + return self._templateClass + + modname = self.opts.templateClassName + + if not modname: + return Template + p = modname.rfind('.') + if ':' not in modname: + self.error('The value of option --templateAPIClass is invalid\n' + 'It must be in the form "module:class", ' + 'e.g. "Cheetah.Template:Template"') + + modname, classname = modname.split(':') + + C('using --templateAPIClass=%s:%s'%(modname, classname)) + + if p >= 0: + mod = getattr(__import__(modname[:p], {}, {}, [modname[p+1:]]), modname[p+1:]) + else: + mod = __import__(modname, {}, {}, []) + + klass = getattr(mod, classname, None) + if klass: + self._templateClass = klass + return klass + else: + self.error('**Template class specified in option --templateAPIClass not found\n' + '**Falling back on Cheetah.Template:Template') + + + def _getCompilerSettings(self): + if self._compilerSettings: + return self._compilerSettings + + def getkws(**kws): + return kws + if self.opts.compilerSettingsString: + try: + exec('settings = getkws(%s)'%self.opts.compilerSettingsString) + except: + self.error("There's an error in your --settings option." + "It must be valid Python syntax.\n" + +" --settings='%s'\n"%self.opts.compilerSettingsString + +" %s: %s"%sys.exc_info()[:2] + ) + + validKeys = DEFAULT_COMPILER_SETTINGS.keys() + if [k for k in settings.keys() if k not in validKeys]: + self.error( + 'The --setting "%s" is not a valid compiler setting name.'%k) + + self._compilerSettings = settings + return settings + else: + return {} + + def _compileOrFillStdin(self): + TemplateClass = self._getTemplateClass() + compilerSettings = self._getCompilerSettings() + if self.isCompile: + pysrc = TemplateClass.compile(file=sys.stdin, + compilerSettings=compilerSettings, + returnAClass=False) + output = pysrc + else: + output = str(TemplateClass(file=sys.stdin, compilerSettings=compilerSettings)) + sys.stdout.write(output) + + def _compileOrFillBundle(self, b): + C, D, W = self.chatter, self.debug, self.warn + TemplateClass = self._getTemplateClass() + compilerSettings = self._getCompilerSettings() + src = b.src + dst = b.dst + base = b.base + basename = b.basename + dstDir = os.path.dirname(dst) + what = self.isCompile and "Compiling" or "Filling" + C("%s %s -> %s^", what, src, dst) # No trailing newline. + if os.path.exists(dst) and not self.opts.nobackup: + bak = b.bak + C(" (backup %s)", bak) # On same line as previous message. + else: + bak = None + C("") + if self.isCompile: + if not moduleNameRE.match(basename): + tup = basename, src + raise Error("""\ +%s: base name %s contains invalid characters. It must +be named according to the same rules as Python modules.""" % tup) + pysrc = TemplateClass.compile(file=src, returnAClass=False, + moduleName=basename, + className=basename, + commandlineopts=self.opts, + compilerSettings=compilerSettings) + output = pysrc + else: + #output = str(TemplateClass(file=src, searchList=self.searchList)) + tclass = TemplateClass.compile(file=src, compilerSettings=compilerSettings) + output = str(tclass(searchList=self.searchList)) + + if bak: + shutil.copyfile(dst, bak) + if dstDir and not os.path.exists(dstDir): + if self.isCompile: + mkdirsWithPyInitFiles(dstDir) + else: + os.makedirs(dstDir) + if self.opts.stdout: + sys.stdout.write(output) + else: + f = open(dst, 'w') + f.write(output) + f.close() + + +# Called when invoked as `cheetah` +def _cheetah(): + CheetahWrapper().main() + +# Called when invoked as `cheetah-compile` +def _cheetah_compile(): + sys.argv.insert(1, "compile") + CheetahWrapper().main() + + +################################################## +## if run from the command line +if __name__ == '__main__': CheetahWrapper().main() + +# vim: shiftwidth=4 tabstop=4 expandtab === added file 'cheetah/Compiler.py' --- cheetah/Compiler.py 1970-01-01 00:00:00 +0000 +++ cheetah/Compiler.py 2010-10-19 19:11:10 +0000 @@ -0,0 +1,2002 @@ +''' + Compiler classes for Cheetah: + ModuleCompiler aka 'Compiler' + ClassCompiler + MethodCompiler + + If you are trying to grok this code start with ModuleCompiler.__init__, + ModuleCompiler.compile, and ModuleCompiler.__getattr__. +''' + +import sys +import os +import os.path +from os.path import getmtime, exists +import re +import types +import time +import random +import warnings +import copy + +from Cheetah.Version import Version, VersionTuple +from Cheetah.SettingsManager import SettingsManager +from Cheetah.Utils.Indenter import indentize # an undocumented preprocessor +from Cheetah import ErrorCatchers +from Cheetah import NameMapper +from Cheetah.Parser import Parser, ParseError, specialVarRE, \ + STATIC_CACHE, REFRESH_CACHE, SET_LOCAL, SET_GLOBAL, SET_MODULE, \ + unicodeDirectiveRE, encodingDirectiveRE, escapedNewlineRE + +from Cheetah.NameMapper import NotFound, valueForName, valueFromSearchList, valueFromFrameOrSearchList +VFFSL=valueFromFrameOrSearchList +VFSL=valueFromSearchList +VFN=valueForName +currentTime=time.time + +class Error(Exception): pass + +# Settings format: (key, default, docstring) +_DEFAULT_COMPILER_SETTINGS = [ + ('useNameMapper', True, 'Enable NameMapper for dotted notation and searchList support'), + ('useSearchList', True, 'Enable the searchList, requires useNameMapper=True, if disabled, first portion of the $variable is a global, builtin, or local variable that doesn\'t need looking up in the searchList'), + ('allowSearchListAsMethArg', True, ''), + ('useAutocalling', True, 'Detect and call callable objects in searchList, requires useNameMapper=True'), + ('useStackFrames', True, 'Used for NameMapper.valueFromFrameOrSearchList rather than NameMapper.valueFromSearchList'), + ('useErrorCatcher', False, 'Turn on the #errorCatcher directive for catching NameMapper errors, etc'), + ('alwaysFilterNone', True, 'Filter out None prior to calling the #filter'), + ('useFilters', True, 'If False, pass output through str()'), + ('includeRawExprInFilterArgs', True, ''), + ('useLegacyImportMode', True, 'All #import statements are relocated to the top of the generated Python module'), + ('prioritizeSearchListOverSelf', False, 'When iterating the searchList, look into the searchList passed into the initializer instead of Template members first'), + + ('autoAssignDummyTransactionToSelf', False, ''), + ('useKWsDictArgForPassingTrans', True, ''), + + ('commentOffset', 1, ''), + ('outputRowColComments', True, ''), + ('includeBlockMarkers', False, 'Wrap #block\'s in a comment in the template\'s output'), + ('blockMarkerStart', ('\n\n'), ''), + ('blockMarkerEnd', ('\n\n'), ''), + ('defDocStrMsg', 'Autogenerated by Cheetah: The Python-Powered Template Engine', ''), + ('setup__str__method', False, ''), + ('mainMethodName', 'respond', ''), + ('mainMethodNameForSubclasses', 'writeBody', ''), + ('indentationStep', ' ' * 4, ''), + ('initialMethIndentLevel', 2, ''), + ('monitorSrcFile', False, ''), + ('outputMethodsBeforeAttributes', True, ''), + ('addTimestampsToCompilerOutput', True, ''), + + ## Customizing the #extends directive + ('autoImportForExtendsDirective', True, ''), + ('handlerForExtendsDirective', None, ''), + + ('disabledDirectives', [], 'List of directive keys to disable (without starting "#")'), + ('enabledDirectives', [], 'List of directive keys to enable (without starting "#")'), + ('disabledDirectiveHooks', [], 'callable(parser, directiveKey)'), + ('preparseDirectiveHooks', [], 'callable(parser, directiveKey)'), + ('postparseDirectiveHooks', [], 'callable(parser, directiveKey)'), + ('preparsePlaceholderHooks', [], 'callable(parser)'), + ('postparsePlaceholderHooks', [], 'callable(parser)'), + ('expressionFilterHooks', [], '''callable(parser, expr, exprType, rawExpr=None, startPos=None), exprType is the name of the directive, "psp" or "placeholder" The filters *must* return the expr or raise an expression, they can modify the expr if needed'''), + ('templateMetaclass', None, 'Strictly optional, only will work with new-style basecalsses as well'), + ('i18NFunctionName', 'self.i18n', ''), + + ('cheetahVarStartToken', '$', ''), + ('commentStartToken', '##', ''), + ('multiLineCommentStartToken', '#*', ''), + ('multiLineCommentEndToken', '*#', ''), + ('gobbleWhitespaceAroundMultiLineComments', True, ''), + ('directiveStartToken', '#', ''), + ('directiveEndToken', '#', ''), + ('allowWhitespaceAfterDirectiveStartToken', False, ''), + ('PSPStartToken', '<%', ''), + ('PSPEndToken', '%>', ''), + ('EOLSlurpToken', '#', ''), + ('gettextTokens', ["_", "N_", "ngettext"], ''), + ('allowExpressionsInExtendsDirective', False, ''), + ('allowEmptySingleLineMethods', False, ''), + ('allowNestedDefScopes', True, ''), + ('allowPlaceholderFilterArgs', True, ''), +] + +DEFAULT_COMPILER_SETTINGS = dict([(v[0], v[1]) for v in _DEFAULT_COMPILER_SETTINGS]) + + + +class GenUtils(object): + """An abstract baseclass for the Compiler classes that provides methods that + perform generic utility functions or generate pieces of output code from + information passed in by the Parser baseclass. These methods don't do any + parsing themselves. + """ + + def genTimeInterval(self, timeString): + ##@@ TR: need to add some error handling here + if timeString[-1] == 's': + interval = float(timeString[:-1]) + elif timeString[-1] == 'm': + interval = float(timeString[:-1])*60 + elif timeString[-1] == 'h': + interval = float(timeString[:-1])*60*60 + elif timeString[-1] == 'd': + interval = float(timeString[:-1])*60*60*24 + elif timeString[-1] == 'w': + interval = float(timeString[:-1])*60*60*24*7 + else: # default to minutes + interval = float(timeString)*60 + return interval + + def genCacheInfo(self, cacheTokenParts): + """Decipher a placeholder cachetoken + """ + cacheInfo = {} + if cacheTokenParts['REFRESH_CACHE']: + cacheInfo['type'] = REFRESH_CACHE + cacheInfo['interval'] = self.genTimeInterval(cacheTokenParts['interval']) + elif cacheTokenParts['STATIC_CACHE']: + cacheInfo['type'] = STATIC_CACHE + return cacheInfo # is empty if no cache + + def genCacheInfoFromArgList(self, argList): + cacheInfo = {'type':REFRESH_CACHE} + for key, val in argList: + if val[0] in '"\'': + val = val[1:-1] + + if key == 'timer': + key = 'interval' + val = self.genTimeInterval(val) + + cacheInfo[key] = val + return cacheInfo + + def genCheetahVar(self, nameChunks, plain=False): + if nameChunks[0][0] in self.setting('gettextTokens'): + self.addGetTextVar(nameChunks) + if self.setting('useNameMapper') and not plain: + return self.genNameMapperVar(nameChunks) + else: + return self.genPlainVar(nameChunks) + + def addGetTextVar(self, nameChunks): + """Output something that gettext can recognize. + + This is a harmless side effect necessary to make gettext work when it + is scanning compiled templates for strings marked for translation. + + @@TR: another marginally more efficient approach would be to put the + output in a dummy method that is never called. + """ + # @@TR: this should be in the compiler not here + self.addChunk("if False:") + self.indent() + self.addChunk(self.genPlainVar(nameChunks[:])) + self.dedent() + + def genPlainVar(self, nameChunks): + """Generate Python code for a Cheetah $var without using NameMapper + (Unified Dotted Notation with the SearchList). + """ + nameChunks.reverse() + chunk = nameChunks.pop() + pythonCode = chunk[0] + chunk[2] + while nameChunks: + chunk = nameChunks.pop() + pythonCode = (pythonCode + '.' + chunk[0] + chunk[2]) + return pythonCode + + def genNameMapperVar(self, nameChunks): + """Generate valid Python code for a Cheetah $var, using NameMapper + (Unified Dotted Notation with the SearchList). + + nameChunks = list of var subcomponents represented as tuples + [ (name,useAC,remainderOfExpr), + ] + where: + name = the dotted name base + useAC = where NameMapper should use autocalling on namemapperPart + remainderOfExpr = any arglist, index, or slice + + If remainderOfExpr contains a call arglist (e.g. '(1234)') then useAC + is False, otherwise it defaults to True. It is overridden by the global + setting 'useAutocalling' if this setting is False. + + EXAMPLE + ------------------------------------------------------------------------ + if the raw Cheetah Var is + $a.b.c[1].d().x.y.z + + nameChunks is the list + [ ('a.b.c',True,'[1]'), # A + ('d',False,'()'), # B + ('x.y.z',True,''), # C + ] + + When this method is fed the list above it returns + VFN(VFN(VFFSL(SL, 'a.b.c',True)[1], 'd',False)(), 'x.y.z',True) + which can be represented as + VFN(B`, name=C[0], executeCallables=(useAC and C[1]))C[2] + where: + VFN = NameMapper.valueForName + VFFSL = NameMapper.valueFromFrameOrSearchList + VFSL = NameMapper.valueFromSearchList # optionally used instead of VFFSL + SL = self.searchList() + useAC = self.setting('useAutocalling') # True in this example + + A = ('a.b.c',True,'[1]') + B = ('d',False,'()') + C = ('x.y.z',True,'') + + C` = VFN( VFN( VFFSL(SL, 'a.b.c',True)[1], + 'd',False)(), + 'x.y.z',True) + = VFN(B`, name='x.y.z', executeCallables=True) + + B` = VFN(A`, name=B[0], executeCallables=(useAC and B[1]))B[2] + A` = VFFSL(SL, name=A[0], executeCallables=(useAC and A[1]))A[2] + + + Note, if the compiler setting useStackFrames=False (default is true) + then + A` = VFSL([locals()]+SL+[globals(), __builtin__], name=A[0], executeCallables=(useAC and A[1]))A[2] + This option allows Cheetah to be used with Psyco, which doesn't support + stack frame introspection. + """ + defaultUseAC = self.setting('useAutocalling') + useSearchList = self.setting('useSearchList') + + nameChunks.reverse() + name, useAC, remainder = nameChunks.pop() + + if not useSearchList: + firstDotIdx = name.find('.') + if firstDotIdx != -1 and firstDotIdx < len(name): + beforeFirstDot, afterDot = name[:firstDotIdx], name[firstDotIdx+1:] + pythonCode = ('VFN(' + beforeFirstDot + + ',"' + afterDot + + '",' + repr(defaultUseAC and useAC) + ')' + + remainder) + else: + pythonCode = name+remainder + elif self.setting('useStackFrames'): + pythonCode = ('VFFSL(SL,' + '"'+ name + '",' + + repr(defaultUseAC and useAC) + ')' + + remainder) + else: + pythonCode = ('VFSL([locals()]+SL+[globals(), builtin],' + '"'+ name + '",' + + repr(defaultUseAC and useAC) + ')' + + remainder) + ## + while nameChunks: + name, useAC, remainder = nameChunks.pop() + pythonCode = ('VFN(' + pythonCode + + ',"' + name + + '",' + repr(defaultUseAC and useAC) + ')' + + remainder) + return pythonCode + +################################################## +## METHOD COMPILERS + +class MethodCompiler(GenUtils): + def __init__(self, methodName, classCompiler, + initialMethodComment=None, + decorators=None): + self._settingsManager = classCompiler + self._classCompiler = classCompiler + self._moduleCompiler = classCompiler._moduleCompiler + self._methodName = methodName + self._initialMethodComment = initialMethodComment + self._setupState() + self._decorators = decorators or [] + + def setting(self, key): + return self._settingsManager.setting(key) + + def _setupState(self): + self._indent = self.setting('indentationStep') + self._indentLev = self.setting('initialMethIndentLevel') + self._pendingStrConstChunks = [] + self._methodSignature = None + self._methodDef = None + self._docStringLines = [] + self._methodBodyChunks = [] + + self._cacheRegionsStack = [] + self._callRegionsStack = [] + self._captureRegionsStack = [] + self._filterRegionsStack = [] + + self._isErrorCatcherOn = False + + self._hasReturnStatement = False + self._isGenerator = False + + + def cleanupState(self): + """Called by the containing class compiler instance + """ + pass + + def methodName(self): + return self._methodName + + def setMethodName(self, name): + self._methodName = name + + ## methods for managing indentation + + def indentation(self): + return self._indent * self._indentLev + + def indent(self): + self._indentLev +=1 + + def dedent(self): + if self._indentLev: + self._indentLev -=1 + else: + raise Error('Attempt to dedent when the indentLev is 0') + + ## methods for final code wrapping + + def methodDef(self): + if self._methodDef: + return self._methodDef + else: + return self.wrapCode() + + __str__ = methodDef + __unicode__ = methodDef + + def wrapCode(self): + self.commitStrConst() + methodDefChunks = ( + self.methodSignature(), + '\n', + self.docString(), + self.methodBody() ) + methodDef = ''.join(methodDefChunks) + self._methodDef = methodDef + return methodDef + + def methodSignature(self): + return self._indent + self._methodSignature + ':' + + def setMethodSignature(self, signature): + self._methodSignature = signature + + def methodBody(self): + return ''.join( self._methodBodyChunks ) + + def docString(self): + if not self._docStringLines: + return '' + + ind = self._indent*2 + docStr = (ind + '"""\n' + ind + + ('\n' + ind).join([ln.replace('"""', "'''") for ln in self._docStringLines]) + + '\n' + ind + '"""\n') + return docStr + + ## methods for adding code + def addMethDocString(self, line): + self._docStringLines.append(line.replace('%', '%%')) + + def addChunk(self, chunk): + self.commitStrConst() + chunk = "\n" + self.indentation() + chunk + self._methodBodyChunks.append(chunk) + + def appendToPrevChunk(self, appendage): + self._methodBodyChunks[-1] = self._methodBodyChunks[-1] + appendage + + def addWriteChunk(self, chunk): + self.addChunk('write(' + chunk + ')') + + def addFilteredChunk(self, chunk, filterArgs=None, rawExpr=None, lineCol=None): + if filterArgs is None: + filterArgs = '' + if self.setting('includeRawExprInFilterArgs') and rawExpr: + filterArgs += ', rawExpr=%s'%repr(rawExpr) + + if self.setting('alwaysFilterNone'): + if rawExpr and rawExpr.find('\n')==-1 and rawExpr.find('\r')==-1: + self.addChunk("_v = %s # %r"%(chunk, rawExpr)) + if lineCol: + self.appendToPrevChunk(' on line %s, col %s'%lineCol) + else: + self.addChunk("_v = %s"%chunk) + + if self.setting('useFilters'): + self.addChunk("if _v is not None: write(_filter(_v%s))"%filterArgs) + else: + self.addChunk("if _v is not None: write(str(_v))") + else: + if self.setting('useFilters'): + self.addChunk("write(_filter(%s%s))"%(chunk, filterArgs)) + else: + self.addChunk("write(str(%s))"%chunk) + + def _appendToPrevStrConst(self, strConst): + if self._pendingStrConstChunks: + self._pendingStrConstChunks.append(strConst) + else: + self._pendingStrConstChunks = [strConst] + + def commitStrConst(self): + """Add the code for outputting the pending strConst without chopping off + any whitespace from it. + """ + if not self._pendingStrConstChunks: + return + + strConst = ''.join(self._pendingStrConstChunks) + self._pendingStrConstChunks = [] + if not strConst: + return + + reprstr = repr(strConst) + i = 0 + out = [] + if reprstr.startswith('u'): + i = 1 + out = ['u'] + body = escapedNewlineRE.sub('\\1\n', reprstr[i+1:-1]) + + if reprstr[i]=="'": + out.append("'''") + out.append(body) + out.append("'''") + else: + out.append('"""') + out.append(body) + out.append('"""') + self.addWriteChunk(''.join(out)) + + def handleWSBeforeDirective(self): + """Truncate the pending strCont to the beginning of the current line. + """ + if self._pendingStrConstChunks: + src = self._pendingStrConstChunks[-1] + BOL = max(src.rfind('\n')+1, src.rfind('\r')+1, 0) + if BOL < len(src): + self._pendingStrConstChunks[-1] = src[:BOL] + + + + def isErrorCatcherOn(self): + return self._isErrorCatcherOn + + def turnErrorCatcherOn(self): + self._isErrorCatcherOn = True + + def turnErrorCatcherOff(self): + self._isErrorCatcherOn = False + + # @@TR: consider merging the next two methods into one + def addStrConst(self, strConst): + self._appendToPrevStrConst(strConst) + + def addRawText(self, text): + self.addStrConst(text) + + def addMethComment(self, comm): + offSet = self.setting('commentOffset') + self.addChunk('#' + ' '*offSet + comm) + + def addPlaceholder(self, expr, filterArgs, rawPlaceholder, + cacheTokenParts, lineCol, + silentMode=False): + cacheInfo = self.genCacheInfo(cacheTokenParts) + if cacheInfo: + cacheInfo['ID'] = repr(rawPlaceholder)[1:-1] + self.startCacheRegion(cacheInfo, lineCol, rawPlaceholder=rawPlaceholder) + + if self.isErrorCatcherOn(): + methodName = self._classCompiler.addErrorCatcherCall( + expr, rawCode=rawPlaceholder, lineCol=lineCol) + expr = 'self.' + methodName + '(localsDict=locals())' + + if silentMode: + self.addChunk('try:') + self.indent() + self.addFilteredChunk(expr, filterArgs, rawPlaceholder, lineCol=lineCol) + self.dedent() + self.addChunk('except NotFound: pass') + else: + self.addFilteredChunk(expr, filterArgs, rawPlaceholder, lineCol=lineCol) + + if self.setting('outputRowColComments'): + self.appendToPrevChunk(' # from line %s, col %s' % lineCol + '.') + if cacheInfo: + self.endCacheRegion() + + def addSilent(self, expr): + self.addChunk( expr ) + + def addEcho(self, expr, rawExpr=None): + self.addFilteredChunk(expr, rawExpr=rawExpr) + + def addSet(self, expr, exprComponents, setStyle): + if setStyle is SET_GLOBAL: + (LVALUE, OP, RVALUE) = (exprComponents.LVALUE, + exprComponents.OP, + exprComponents.RVALUE) + # we need to split the LVALUE to deal with globalSetVars + splitPos1 = LVALUE.find('.') + splitPos2 = LVALUE.find('[') + if splitPos1 > 0 and splitPos2==-1: + splitPos = splitPos1 + elif splitPos1 > 0 and splitPos1 < max(splitPos2, 0): + splitPos = splitPos1 + else: + splitPos = splitPos2 + + if splitPos >0: + primary = LVALUE[:splitPos] + secondary = LVALUE[splitPos:] + else: + primary = LVALUE + secondary = '' + LVALUE = 'self._CHEETAH__globalSetVars["' + primary + '"]' + secondary + expr = LVALUE + ' ' + OP + ' ' + RVALUE.strip() + + if setStyle is SET_MODULE: + self._moduleCompiler.addModuleGlobal(expr) + else: + self.addChunk(expr) + + def addInclude(self, sourceExpr, includeFrom, isRaw): + self.addChunk('self._handleCheetahInclude(' + sourceExpr + + ', trans=trans, ' + + 'includeFrom="' + includeFrom + '", raw=' + + repr(isRaw) + ')') + + def addWhile(self, expr, lineCol=None): + self.addIndentingDirective(expr, lineCol=lineCol) + + def addFor(self, expr, lineCol=None): + self.addIndentingDirective(expr, lineCol=lineCol) + + def addRepeat(self, expr, lineCol=None): + #the _repeatCount stuff here allows nesting of #repeat directives + self._repeatCount = getattr(self, "_repeatCount", -1) + 1 + self.addFor('for __i%s in range(%s)' % (self._repeatCount, expr), lineCol=lineCol) + + def addIndentingDirective(self, expr, lineCol=None): + if expr and not expr[-1] == ':': + expr = expr + ':' + self.addChunk( expr ) + if lineCol: + self.appendToPrevChunk(' # generated from line %s, col %s'%lineCol ) + self.indent() + + def addReIndentingDirective(self, expr, dedent=True, lineCol=None): + self.commitStrConst() + if dedent: + self.dedent() + if not expr[-1] == ':': + expr = expr + ':' + + self.addChunk( expr ) + if lineCol: + self.appendToPrevChunk(' # generated from line %s, col %s'%lineCol ) + self.indent() + + def addIf(self, expr, lineCol=None): + """For a full #if ... #end if directive + """ + self.addIndentingDirective(expr, lineCol=lineCol) + + def addOneLineIf(self, expr, lineCol=None): + """For a full #if ... #end if directive + """ + self.addIndentingDirective(expr, lineCol=lineCol) + + def addTernaryExpr(self, conditionExpr, trueExpr, falseExpr, lineCol=None): + """For a single-lie #if ... then .... else ... directive + then else + """ + self.addIndentingDirective(conditionExpr, lineCol=lineCol) + self.addFilteredChunk(trueExpr) + self.dedent() + self.addIndentingDirective('else') + self.addFilteredChunk(falseExpr) + self.dedent() + + def addElse(self, expr, dedent=True, lineCol=None): + expr = re.sub(r'else[ \f\t]+if', 'elif', expr) + self.addReIndentingDirective(expr, dedent=dedent, lineCol=lineCol) + + def addElif(self, expr, dedent=True, lineCol=None): + self.addElse(expr, dedent=dedent, lineCol=lineCol) + + def addUnless(self, expr, lineCol=None): + self.addIf('if not (' + expr + ')') + + def addClosure(self, functionName, argsList, parserComment): + argStringChunks = [] + for arg in argsList: + chunk = arg[0] + if not arg[1] == None: + chunk += '=' + arg[1] + argStringChunks.append(chunk) + signature = "def " + functionName + "(" + ','.join(argStringChunks) + "):" + self.addIndentingDirective(signature) + self.addChunk('#'+parserComment) + + def addTry(self, expr, lineCol=None): + self.addIndentingDirective(expr, lineCol=lineCol) + + def addExcept(self, expr, dedent=True, lineCol=None): + self.addReIndentingDirective(expr, dedent=dedent, lineCol=lineCol) + + def addFinally(self, expr, dedent=True, lineCol=None): + self.addReIndentingDirective(expr, dedent=dedent, lineCol=lineCol) + + def addReturn(self, expr): + assert not self._isGenerator + self.addChunk(expr) + self._hasReturnStatement = True + + def addYield(self, expr): + assert not self._hasReturnStatement + self._isGenerator = True + if expr.replace('yield', '').strip(): + self.addChunk(expr) + else: + self.addChunk('if _dummyTrans:') + self.indent() + self.addChunk('yield trans.response().getvalue()') + self.addChunk('trans = DummyTransaction()') + self.addChunk('write = trans.response().write') + self.dedent() + self.addChunk('else:') + self.indent() + self.addChunk( + 'raise TypeError("This method cannot be called with a trans arg")') + self.dedent() + + + def addPass(self, expr): + self.addChunk(expr) + + def addDel(self, expr): + self.addChunk(expr) + + def addAssert(self, expr): + self.addChunk(expr) + + def addRaise(self, expr): + self.addChunk(expr) + + def addBreak(self, expr): + self.addChunk(expr) + + def addContinue(self, expr): + self.addChunk(expr) + + def addPSP(self, PSP): + self.commitStrConst() + autoIndent = False + if PSP[0] == '=': + PSP = PSP[1:] + if PSP: + self.addWriteChunk('_filter(' + PSP + ')') + return + + elif PSP.lower() == 'end': + self.dedent() + return + elif PSP[-1] == '$': + autoIndent = True + PSP = PSP[:-1] + elif PSP[-1] == ':': + autoIndent = True + + for line in PSP.splitlines(): + self.addChunk(line) + + if autoIndent: + self.indent() + + def nextCacheID(self): + return ('_'+str(random.randrange(100, 999)) + + str(random.randrange(10000, 99999))) + + def startCacheRegion(self, cacheInfo, lineCol, rawPlaceholder=None): + + # @@TR: we should add some runtime logging to this + + ID = self.nextCacheID() + interval = cacheInfo.get('interval', None) + test = cacheInfo.get('test', None) + customID = cacheInfo.get('id', None) + if customID: + ID = customID + varyBy = cacheInfo.get('varyBy', repr(ID)) + self._cacheRegionsStack.append(ID) # attrib of current methodCompiler + + # @@TR: add this to a special class var as well + self.addChunk('') + + self.addChunk('## START CACHE REGION: ID='+ID+ + '. line %s, col %s'%lineCol + ' in the source.') + + self.addChunk('_RECACHE_%(ID)s = False'%locals()) + self.addChunk('_cacheRegion_%(ID)s = self.getCacheRegion(regionID='%locals() + + repr(ID) + + ', cacheInfo=%r'%cacheInfo + + ')') + self.addChunk('if _cacheRegion_%(ID)s.isNew():'%locals()) + self.indent() + self.addChunk('_RECACHE_%(ID)s = True'%locals()) + self.dedent() + + self.addChunk('_cacheItem_%(ID)s = _cacheRegion_%(ID)s.getCacheItem('%locals() + +varyBy+')') + + self.addChunk('if _cacheItem_%(ID)s.hasExpired():'%locals()) + self.indent() + self.addChunk('_RECACHE_%(ID)s = True'%locals()) + self.dedent() + + if test: + self.addChunk('if ' + test + ':') + self.indent() + self.addChunk('_RECACHE_%(ID)s = True'%locals()) + self.dedent() + + self.addChunk('if (not _RECACHE_%(ID)s) and _cacheItem_%(ID)s.getRefreshTime():'%locals()) + self.indent() + #self.addChunk('print "DEBUG"+"-"*50') + self.addChunk('try:') + self.indent() + self.addChunk('_output = _cacheItem_%(ID)s.renderOutput()'%locals()) + self.dedent() + self.addChunk('except KeyError:') + self.indent() + self.addChunk('_RECACHE_%(ID)s = True'%locals()) + #self.addChunk('print "DEBUG"+"*"*50') + self.dedent() + self.addChunk('else:') + self.indent() + self.addWriteChunk('_output') + self.addChunk('del _output') + self.dedent() + + self.dedent() + + self.addChunk('if _RECACHE_%(ID)s or not _cacheItem_%(ID)s.getRefreshTime():'%locals()) + self.indent() + self.addChunk('_orig_trans%(ID)s = trans'%locals()) + self.addChunk('trans = _cacheCollector_%(ID)s = DummyTransaction()'%locals()) + self.addChunk('write = _cacheCollector_%(ID)s.response().write'%locals()) + if interval: + self.addChunk(("_cacheItem_%(ID)s.setExpiryTime(currentTime() +"%locals()) + + str(interval) + ")") + + def endCacheRegion(self): + ID = self._cacheRegionsStack.pop() + self.addChunk('trans = _orig_trans%(ID)s'%locals()) + self.addChunk('write = trans.response().write') + self.addChunk('_cacheData = _cacheCollector_%(ID)s.response().getvalue()'%locals()) + self.addChunk('_cacheItem_%(ID)s.setData(_cacheData)'%locals()) + self.addWriteChunk('_cacheData') + self.addChunk('del _cacheData') + self.addChunk('del _cacheCollector_%(ID)s'%locals()) + self.addChunk('del _orig_trans%(ID)s'%locals()) + self.dedent() + self.addChunk('## END CACHE REGION: '+ID) + self.addChunk('') + + def nextCallRegionID(self): + return self.nextCacheID() + + def startCallRegion(self, functionName, args, lineCol, regionTitle='CALL'): + class CallDetails(object): + pass + callDetails = CallDetails() + callDetails.ID = ID = self.nextCallRegionID() + callDetails.functionName = functionName + callDetails.args = args + callDetails.lineCol = lineCol + callDetails.usesKeywordArgs = False + self._callRegionsStack.append((ID, callDetails)) # attrib of current methodCompiler + + self.addChunk('## START %(regionTitle)s REGION: '%locals() + +ID + +' of '+functionName + +' at line %s, col %s'%lineCol + ' in the source.') + self.addChunk('_orig_trans%(ID)s = trans'%locals()) + self.addChunk('_wasBuffering%(ID)s = self._CHEETAH__isBuffering'%locals()) + self.addChunk('self._CHEETAH__isBuffering = True') + self.addChunk('trans = _callCollector%(ID)s = DummyTransaction()'%locals()) + self.addChunk('write = _callCollector%(ID)s.response().write'%locals()) + + def setCallArg(self, argName, lineCol): + ID, callDetails = self._callRegionsStack[-1] + argName = str(argName) + if callDetails.usesKeywordArgs: + self._endCallArg() + else: + callDetails.usesKeywordArgs = True + self.addChunk('_callKws%(ID)s = {}'%locals()) + self.addChunk('_currentCallArgname%(ID)s = %(argName)r'%locals()) + callDetails.currentArgname = argName + + def _endCallArg(self): + ID, callDetails = self._callRegionsStack[-1] + currCallArg = callDetails.currentArgname + self.addChunk(('_callKws%(ID)s[%(currCallArg)r] =' + ' _callCollector%(ID)s.response().getvalue()')%locals()) + self.addChunk('del _callCollector%(ID)s'%locals()) + self.addChunk('trans = _callCollector%(ID)s = DummyTransaction()'%locals()) + self.addChunk('write = _callCollector%(ID)s.response().write'%locals()) + + def endCallRegion(self, regionTitle='CALL'): + ID, callDetails = self._callRegionsStack[-1] + functionName, initialKwArgs, lineCol = ( + callDetails.functionName, callDetails.args, callDetails.lineCol) + + def reset(ID=ID): + self.addChunk('trans = _orig_trans%(ID)s'%locals()) + self.addChunk('write = trans.response().write') + self.addChunk('self._CHEETAH__isBuffering = _wasBuffering%(ID)s '%locals()) + self.addChunk('del _wasBuffering%(ID)s'%locals()) + self.addChunk('del _orig_trans%(ID)s'%locals()) + + if not callDetails.usesKeywordArgs: + reset() + self.addChunk('_callArgVal%(ID)s = _callCollector%(ID)s.response().getvalue()'%locals()) + self.addChunk('del _callCollector%(ID)s'%locals()) + if initialKwArgs: + initialKwArgs = ', '+initialKwArgs + self.addFilteredChunk('%(functionName)s(_callArgVal%(ID)s%(initialKwArgs)s)'%locals()) + self.addChunk('del _callArgVal%(ID)s'%locals()) + else: + if initialKwArgs: + initialKwArgs = initialKwArgs+', ' + self._endCallArg() + reset() + self.addFilteredChunk('%(functionName)s(%(initialKwArgs)s**_callKws%(ID)s)'%locals()) + self.addChunk('del _callKws%(ID)s'%locals()) + self.addChunk('## END %(regionTitle)s REGION: '%locals() + +ID + +' of '+functionName + +' at line %s, col %s'%lineCol + ' in the source.') + self.addChunk('') + self._callRegionsStack.pop() # attrib of current methodCompiler + + def nextCaptureRegionID(self): + return self.nextCacheID() + + def startCaptureRegion(self, assignTo, lineCol): + class CaptureDetails: pass + captureDetails = CaptureDetails() + captureDetails.ID = ID = self.nextCaptureRegionID() + captureDetails.assignTo = assignTo + captureDetails.lineCol = lineCol + + self._captureRegionsStack.append((ID, captureDetails)) # attrib of current methodCompiler + self.addChunk('## START CAPTURE REGION: '+ID + +' '+assignTo + +' at line %s, col %s'%lineCol + ' in the source.') + self.addChunk('_orig_trans%(ID)s = trans'%locals()) + self.addChunk('_wasBuffering%(ID)s = self._CHEETAH__isBuffering'%locals()) + self.addChunk('self._CHEETAH__isBuffering = True') + self.addChunk('trans = _captureCollector%(ID)s = DummyTransaction()'%locals()) + self.addChunk('write = _captureCollector%(ID)s.response().write'%locals()) + + def endCaptureRegion(self): + ID, captureDetails = self._captureRegionsStack.pop() + assignTo, lineCol = (captureDetails.assignTo, captureDetails.lineCol) + self.addChunk('trans = _orig_trans%(ID)s'%locals()) + self.addChunk('write = trans.response().write') + self.addChunk('self._CHEETAH__isBuffering = _wasBuffering%(ID)s '%locals()) + self.addChunk('%(assignTo)s = _captureCollector%(ID)s.response().getvalue()'%locals()) + self.addChunk('del _orig_trans%(ID)s'%locals()) + self.addChunk('del _captureCollector%(ID)s'%locals()) + self.addChunk('del _wasBuffering%(ID)s'%locals()) + + def setErrorCatcher(self, errorCatcherName): + self.turnErrorCatcherOn() + + self.addChunk('if self._CHEETAH__errorCatchers.has_key("' + errorCatcherName + '"):') + self.indent() + self.addChunk('self._CHEETAH__errorCatcher = self._CHEETAH__errorCatchers["' + + errorCatcherName + '"]') + self.dedent() + self.addChunk('else:') + self.indent() + self.addChunk('self._CHEETAH__errorCatcher = self._CHEETAH__errorCatchers["' + + errorCatcherName + '"] = ErrorCatchers.' + + errorCatcherName + '(self)' + ) + self.dedent() + + def nextFilterRegionID(self): + return self.nextCacheID() + + def setTransform(self, transformer, isKlass): + self.addChunk('trans = TransformerTransaction()') + self.addChunk('trans._response = trans.response()') + self.addChunk('trans._response._filter = %s' % transformer) + self.addChunk('write = trans._response.write') + + def setFilter(self, theFilter, isKlass): + class FilterDetails: + pass + filterDetails = FilterDetails() + filterDetails.ID = ID = self.nextFilterRegionID() + filterDetails.theFilter = theFilter + filterDetails.isKlass = isKlass + self._filterRegionsStack.append((ID, filterDetails)) # attrib of current methodCompiler + + self.addChunk('_orig_filter%(ID)s = _filter'%locals()) + if isKlass: + self.addChunk('_filter = self._CHEETAH__currentFilter = ' + theFilter.strip() + + '(self).filter') + else: + if theFilter.lower() == 'none': + self.addChunk('_filter = self._CHEETAH__initialFilter') + else: + # is string representing the name of a builtin filter + self.addChunk('filterName = ' + repr(theFilter)) + self.addChunk('if self._CHEETAH__filters.has_key("' + theFilter + '"):') + self.indent() + self.addChunk('_filter = self._CHEETAH__currentFilter = self._CHEETAH__filters[filterName]') + self.dedent() + self.addChunk('else:') + self.indent() + self.addChunk('_filter = self._CHEETAH__currentFilter' + +' = \\\n\t\t\tself._CHEETAH__filters[filterName] = ' + + 'getattr(self._CHEETAH__filtersLib, filterName)(self).filter') + self.dedent() + + def closeFilterBlock(self): + ID, filterDetails = self._filterRegionsStack.pop() + #self.addChunk('_filter = self._CHEETAH__initialFilter') + #self.addChunk('_filter = _orig_filter%(ID)s'%locals()) + self.addChunk('_filter = self._CHEETAH__currentFilter = _orig_filter%(ID)s'%locals()) + +class AutoMethodCompiler(MethodCompiler): + + def _setupState(self): + MethodCompiler._setupState(self) + self._argStringList = [ ("self", None) ] + self._streamingEnabled = True + self._isClassMethod = None + self._isStaticMethod = None + + def _useKWsDictArgForPassingTrans(self): + alreadyHasTransArg = [argname for argname, defval in self._argStringList + if argname=='trans'] + return (self.methodName()!='respond' + and not alreadyHasTransArg + and self.setting('useKWsDictArgForPassingTrans')) + + def isClassMethod(self): + if self._isClassMethod is None: + self._isClassMethod = '@classmethod' in self._decorators + return self._isClassMethod + + def isStaticMethod(self): + if self._isStaticMethod is None: + self._isStaticMethod = '@staticmethod' in self._decorators + return self._isStaticMethod + + def cleanupState(self): + MethodCompiler.cleanupState(self) + self.commitStrConst() + if self._cacheRegionsStack: + self.endCacheRegion() + if self._callRegionsStack: + self.endCallRegion() + + if self._streamingEnabled: + kwargsName = None + positionalArgsListName = None + for argname, defval in self._argStringList: + if argname.strip().startswith('**'): + kwargsName = argname.strip().replace('**', '') + break + elif argname.strip().startswith('*'): + positionalArgsListName = argname.strip().replace('*', '') + + if not kwargsName and self._useKWsDictArgForPassingTrans(): + kwargsName = 'KWS' + self.addMethArg('**KWS', None) + self._kwargsName = kwargsName + + if not self._useKWsDictArgForPassingTrans(): + if not kwargsName and not positionalArgsListName: + self.addMethArg('trans', 'None') + else: + self._streamingEnabled = False + + self._indentLev = self.setting('initialMethIndentLevel') + mainBodyChunks = self._methodBodyChunks + self._methodBodyChunks = [] + self._addAutoSetupCode() + self._methodBodyChunks.extend(mainBodyChunks) + self._addAutoCleanupCode() + + def _addAutoSetupCode(self): + if self._initialMethodComment: + self.addChunk(self._initialMethodComment) + + if self._streamingEnabled and not self.isClassMethod() and not self.isStaticMethod(): + if self._useKWsDictArgForPassingTrans() and self._kwargsName: + self.addChunk('trans = %s.get("trans")'%self._kwargsName) + self.addChunk('if (not trans and not self._CHEETAH__isBuffering' + ' and not callable(self.transaction)):') + self.indent() + self.addChunk('trans = self.transaction' + ' # is None unless self.awake() was called') + self.dedent() + self.addChunk('if not trans:') + self.indent() + self.addChunk('trans = DummyTransaction()') + if self.setting('autoAssignDummyTransactionToSelf'): + self.addChunk('self.transaction = trans') + self.addChunk('_dummyTrans = True') + self.dedent() + self.addChunk('else: _dummyTrans = False') + else: + self.addChunk('trans = DummyTransaction()') + self.addChunk('_dummyTrans = True') + self.addChunk('write = trans.response().write') + if self.setting('useNameMapper'): + argNames = [arg[0] for arg in self._argStringList] + allowSearchListAsMethArg = self.setting('allowSearchListAsMethArg') + if allowSearchListAsMethArg and 'SL' in argNames: + pass + elif allowSearchListAsMethArg and 'searchList' in argNames: + self.addChunk('SL = searchList') + elif not self.isClassMethod() and not self.isStaticMethod(): + self.addChunk('SL = self._CHEETAH__searchList') + else: + self.addChunk('SL = [KWS]') + if self.setting('useFilters'): + if self.isClassMethod() or self.isStaticMethod(): + self.addChunk('_filter = lambda x, **kwargs: unicode(x)') + else: + self.addChunk('_filter = self._CHEETAH__currentFilter') + self.addChunk('') + self.addChunk("#" *40) + self.addChunk('## START - generated method body') + self.addChunk('') + + def _addAutoCleanupCode(self): + self.addChunk('') + self.addChunk("#" *40) + self.addChunk('## END - generated method body') + self.addChunk('') + + if not self._isGenerator: + self.addStop() + self.addChunk('') + + def addStop(self, expr=None): + self.addChunk('return _dummyTrans and trans.response().getvalue() or ""') + + def addMethArg(self, name, defVal=None): + self._argStringList.append( (name, defVal) ) + + def methodSignature(self): + argStringChunks = [] + for arg in self._argStringList: + chunk = arg[0] + if chunk == 'self' and self.isClassMethod(): + chunk = 'cls' + if chunk == 'self' and self.isStaticMethod(): + # Skip the "self" method for @staticmethod decorators + continue + if not arg[1] == None: + chunk += '=' + arg[1] + argStringChunks.append(chunk) + argString = (', ').join(argStringChunks) + + output = [] + if self._decorators: + output.append(''.join([self._indent + decorator + '\n' + for decorator in self._decorators])) + output.append(self._indent + "def " + + self.methodName() + "(" + + argString + "):\n\n") + return ''.join(output) + + +################################################## +## CLASS COMPILERS + +_initMethod_initCheetah = """\ +if not self._CHEETAH__instanceInitialized: + cheetahKWArgs = {} + allowedKWs = 'searchList namespaces filter filtersLib errorCatcher'.split() + for k,v in KWs.items(): + if k in allowedKWs: cheetahKWArgs[k] = v + self._initCheetahInstance(**cheetahKWArgs) +""".replace('\n', '\n'+' '*8) + +class ClassCompiler(GenUtils): + methodCompilerClass = AutoMethodCompiler + methodCompilerClassForInit = MethodCompiler + + def __init__(self, className, mainMethodName='respond', + moduleCompiler=None, + fileName=None, + settingsManager=None): + + self._settingsManager = settingsManager + self._fileName = fileName + self._className = className + self._moduleCompiler = moduleCompiler + self._mainMethodName = mainMethodName + self._setupState() + methodCompiler = self._spawnMethodCompiler( + mainMethodName, + initialMethodComment='## CHEETAH: main method generated for this template') + + self._setActiveMethodCompiler(methodCompiler) + if fileName and self.setting('monitorSrcFile'): + self._addSourceFileMonitoring(fileName) + + def setting(self, key): + return self._settingsManager.setting(key) + + def __getattr__(self, name): + """Provide access to the methods and attributes of the MethodCompiler + at the top of the activeMethods stack: one-way namespace sharing + + + WARNING: Use .setMethods to assign the attributes of the MethodCompiler + from the methods of this class!!! or you will be assigning to attributes + of this object instead.""" + + if name in self.__dict__: + return self.__dict__[name] + elif hasattr(self.__class__, name): + return getattr(self.__class__, name) + elif self._activeMethodsList and hasattr(self._activeMethodsList[-1], name): + return getattr(self._activeMethodsList[-1], name) + else: + raise AttributeError(name) + + def _setupState(self): + self._classDef = None + self._decoratorsForNextMethod = [] + self._activeMethodsList = [] # stack while parsing/generating + self._finishedMethodsList = [] # store by order + self._methodsIndex = {} # store by name + self._baseClass = 'Template' + self._classDocStringLines = [] + # printed after methods in the gen class def: + self._generatedAttribs = ['_CHEETAH__instanceInitialized = False'] + self._generatedAttribs.append('_CHEETAH_version = __CHEETAH_version__') + self._generatedAttribs.append( + '_CHEETAH_versionTuple = __CHEETAH_versionTuple__') + + if self.setting('addTimestampsToCompilerOutput'): + self._generatedAttribs.append('_CHEETAH_genTime = __CHEETAH_genTime__') + self._generatedAttribs.append('_CHEETAH_genTimestamp = __CHEETAH_genTimestamp__') + + self._generatedAttribs.append('_CHEETAH_src = __CHEETAH_src__') + self._generatedAttribs.append( + '_CHEETAH_srcLastModified = __CHEETAH_srcLastModified__') + + if self.setting('templateMetaclass'): + self._generatedAttribs.append('__metaclass__ = '+self.setting('templateMetaclass')) + self._initMethChunks = [] + self._blockMetaData = {} + self._errorCatcherCount = 0 + self._placeholderToErrorCatcherMap = {} + + def cleanupState(self): + while self._activeMethodsList: + methCompiler = self._popActiveMethodCompiler() + self._swallowMethodCompiler(methCompiler) + self._setupInitMethod() + if self._mainMethodName == 'respond': + if self.setting('setup__str__method'): + self._generatedAttribs.append('def __str__(self): return self.respond()') + self.addAttribute('_mainCheetahMethod_for_' + self._className + + '= ' + repr(self._mainMethodName) ) + + def _setupInitMethod(self): + __init__ = self._spawnMethodCompiler('__init__', + klass=self.methodCompilerClassForInit) + __init__.setMethodSignature("def __init__(self, *args, **KWs)") + __init__.addChunk('super(%s, self).__init__(*args, **KWs)' % self._className) + __init__.addChunk(_initMethod_initCheetah % {'className' : self._className}) + for chunk in self._initMethChunks: + __init__.addChunk(chunk) + __init__.cleanupState() + self._swallowMethodCompiler(__init__, pos=0) + + def _addSourceFileMonitoring(self, fileName): + # @@TR: this stuff needs auditing for Cheetah 2.0 + # the first bit is added to init + self.addChunkToInit('self._filePath = ' + repr(fileName)) + self.addChunkToInit('self._fileMtime = ' + str(getmtime(fileName)) ) + + # the rest is added to the main output method of the class ('mainMethod') + self.addChunk('if exists(self._filePath) and ' + + 'getmtime(self._filePath) > self._fileMtime:') + self.indent() + self.addChunk('self._compile(file=self._filePath, moduleName='+self._className + ')') + self.addChunk( + 'write(getattr(self, self._mainCheetahMethod_for_' + self._className + + ')(trans=trans))') + self.addStop() + self.dedent() + + def setClassName(self, name): + self._className = name + + def className(self): + return self._className + + def setBaseClass(self, baseClassName): + self._baseClass = baseClassName + + def setMainMethodName(self, methodName): + if methodName == self._mainMethodName: + return + ## change the name in the methodCompiler and add new reference + mainMethod = self._methodsIndex[self._mainMethodName] + mainMethod.setMethodName(methodName) + self._methodsIndex[methodName] = mainMethod + + ## make sure that fileUpdate code still works properly: + chunkToChange = ('write(self.' + self._mainMethodName + '(trans=trans))') + chunks = mainMethod._methodBodyChunks + if chunkToChange in chunks: + for i in range(len(chunks)): + if chunks[i] == chunkToChange: + chunks[i] = ('write(self.' + methodName + '(trans=trans))') + ## get rid of the old reference and update self._mainMethodName + del self._methodsIndex[self._mainMethodName] + self._mainMethodName = methodName + + def setMainMethodArgs(self, argsList): + mainMethodCompiler = self._methodsIndex[self._mainMethodName] + for argName, defVal in argsList: + mainMethodCompiler.addMethArg(argName, defVal) + + + def _spawnMethodCompiler(self, methodName, klass=None, + initialMethodComment=None): + if klass is None: + klass = self.methodCompilerClass + + decorators = self._decoratorsForNextMethod or [] + self._decoratorsForNextMethod = [] + methodCompiler = klass(methodName, classCompiler=self, + decorators=decorators, + initialMethodComment=initialMethodComment) + self._methodsIndex[methodName] = methodCompiler + return methodCompiler + + def _setActiveMethodCompiler(self, methodCompiler): + self._activeMethodsList.append(methodCompiler) + + def _getActiveMethodCompiler(self): + return self._activeMethodsList[-1] + + def _popActiveMethodCompiler(self): + return self._activeMethodsList.pop() + + def _swallowMethodCompiler(self, methodCompiler, pos=None): + methodCompiler.cleanupState() + if pos==None: + self._finishedMethodsList.append( methodCompiler ) + else: + self._finishedMethodsList.insert(pos, methodCompiler) + return methodCompiler + + def startMethodDef(self, methodName, argsList, parserComment): + methodCompiler = self._spawnMethodCompiler( + methodName, initialMethodComment=parserComment) + self._setActiveMethodCompiler(methodCompiler) + for argName, defVal in argsList: + methodCompiler.addMethArg(argName, defVal) + + def _finishedMethods(self): + return self._finishedMethodsList + + def addDecorator(self, decoratorExpr): + """Set the decorator to be used with the next method in the source. + + See _spawnMethodCompiler() and MethodCompiler for the details of how + this is used. + """ + self._decoratorsForNextMethod.append(decoratorExpr) + + def addClassDocString(self, line): + self._classDocStringLines.append( line.replace('%', '%%')) + + def addChunkToInit(self, chunk): + self._initMethChunks.append(chunk) + + def addAttribute(self, attribExpr): + ## first test to make sure that the user hasn't used any fancy Cheetah syntax + # (placeholders, directives, etc.) inside the expression + if attribExpr.find('VFN(') != -1 or attribExpr.find('VFFSL(') != -1: + raise ParseError(self, + 'Invalid #attr directive.' + + ' It should only contain simple Python literals.') + ## now add the attribute + self._generatedAttribs.append(attribExpr) + + def addSuper(self, argsList, parserComment=None): + className = self._className #self._baseClass + methodName = self._getActiveMethodCompiler().methodName() + + argStringChunks = [] + for arg in argsList: + chunk = arg[0] + if not arg[1] == None: + chunk += '=' + arg[1] + argStringChunks.append(chunk) + argString = ','.join(argStringChunks) + + self.addFilteredChunk( + 'super(%(className)s, self).%(methodName)s(%(argString)s)'%locals()) + + def addErrorCatcherCall(self, codeChunk, rawCode='', lineCol=''): + if rawCode in self._placeholderToErrorCatcherMap: + methodName = self._placeholderToErrorCatcherMap[rawCode] + if not self.setting('outputRowColComments'): + self._methodsIndex[methodName].addMethDocString( + 'plus at line %s, col %s'%lineCol) + return methodName + + self._errorCatcherCount += 1 + methodName = '__errorCatcher' + str(self._errorCatcherCount) + self._placeholderToErrorCatcherMap[rawCode] = methodName + + catcherMeth = self._spawnMethodCompiler( + methodName, + klass=MethodCompiler, + initialMethodComment=('## CHEETAH: Generated from ' + rawCode + + ' at line %s, col %s'%lineCol + '.') + ) + catcherMeth.setMethodSignature('def ' + methodName + + '(self, localsDict={})') + # is this use of localsDict right? + catcherMeth.addChunk('try:') + catcherMeth.indent() + catcherMeth.addChunk("return eval('''" + codeChunk + + "''', globals(), localsDict)") + catcherMeth.dedent() + catcherMeth.addChunk('except self._CHEETAH__errorCatcher.exceptions(), e:') + catcherMeth.indent() + catcherMeth.addChunk("return self._CHEETAH__errorCatcher.warn(exc_val=e, code= " + + repr(codeChunk) + " , rawCode= " + + repr(rawCode) + " , lineCol=" + str(lineCol) +")") + + catcherMeth.cleanupState() + + self._swallowMethodCompiler(catcherMeth) + return methodName + + def closeDef(self): + self.commitStrConst() + methCompiler = self._popActiveMethodCompiler() + self._swallowMethodCompiler(methCompiler) + + def closeBlock(self): + self.commitStrConst() + methCompiler = self._popActiveMethodCompiler() + methodName = methCompiler.methodName() + if self.setting('includeBlockMarkers'): + endMarker = self.setting('blockMarkerEnd') + methCompiler.addStrConst(endMarker[0] + methodName + endMarker[1]) + self._swallowMethodCompiler(methCompiler) + + #metaData = self._blockMetaData[methodName] + #rawDirective = metaData['raw'] + #lineCol = metaData['lineCol'] + + ## insert the code to call the block, caching if #cache directive is on + codeChunk = 'self.' + methodName + '(trans=trans)' + self.addChunk(codeChunk) + + #self.appendToPrevChunk(' # generated from ' + repr(rawDirective) ) + #if self.setting('outputRowColComments'): + # self.appendToPrevChunk(' at line %s, col %s' % lineCol + '.') + + + ## code wrapping methods + + def classDef(self): + if self._classDef: + return self._classDef + else: + return self.wrapClassDef() + + __str__ = classDef + __unicode__ = classDef + + def wrapClassDef(self): + ind = self.setting('indentationStep') + classDefChunks = [self.classSignature(), + self.classDocstring(), + ] + def addMethods(): + classDefChunks.extend([ + ind + '#'*50, + ind + '## CHEETAH GENERATED METHODS', + '\n', + self.methodDefs(), + ]) + def addAttributes(): + classDefChunks.extend([ + ind + '#'*50, + ind + '## CHEETAH GENERATED ATTRIBUTES', + '\n', + self.attributes(), + ]) + if self.setting('outputMethodsBeforeAttributes'): + addMethods() + addAttributes() + else: + addAttributes() + addMethods() + + classDef = '\n'.join(classDefChunks) + self._classDef = classDef + return classDef + + + def classSignature(self): + return "class %s(%s):" % (self.className(), self._baseClass) + + def classDocstring(self): + if not self._classDocStringLines: + return '' + ind = self.setting('indentationStep') + docStr = ('%(ind)s"""\n%(ind)s' + + '\n%(ind)s'.join(self._classDocStringLines) + + '\n%(ind)s"""\n' + ) % {'ind':ind} + return docStr + + def methodDefs(self): + methodDefs = [methGen.methodDef() for methGen in self._finishedMethods()] + return '\n\n'.join(methodDefs) + + def attributes(self): + attribs = [self.setting('indentationStep') + str(attrib) + for attrib in self._generatedAttribs ] + return '\n\n'.join(attribs) + +class AutoClassCompiler(ClassCompiler): + pass + +################################################## +## MODULE COMPILERS + +class ModuleCompiler(SettingsManager, GenUtils): + + parserClass = Parser + classCompilerClass = AutoClassCompiler + + def __init__(self, source=None, file=None, + moduleName='DynamicallyCompiledCheetahTemplate', + mainClassName=None, # string + mainMethodName=None, # string + baseclassName=None, # string + extraImportStatements=None, # list of strings + settings=None # dict + ): + super(ModuleCompiler, self).__init__() + if settings: + self.updateSettings(settings) + # disable useStackFrames if the C version of NameMapper isn't compiled + # it's painfully slow in the Python version and bites Windows users all + # the time: + if not NameMapper.C_VERSION: + if not sys.platform.startswith('java'): + warnings.warn( + "\nYou don't have the C version of NameMapper installed! " + "I'm disabling Cheetah's useStackFrames option as it is " + "painfully slow with the Python version of NameMapper. " + "You should get a copy of Cheetah with the compiled C version of NameMapper." + ) + self.setSetting('useStackFrames', False) + + self._compiled = False + self._moduleName = moduleName + if not mainClassName: + self._mainClassName = moduleName + else: + self._mainClassName = mainClassName + self._mainMethodNameArg = mainMethodName + if mainMethodName: + self.setSetting('mainMethodName', mainMethodName) + self._baseclassName = baseclassName + + self._filePath = None + self._fileMtime = None + + if source and file: + raise TypeError("Cannot compile from a source string AND file.") + elif isinstance(file, basestring): # it's a filename. + f = open(file) # Raises IOError. + source = f.read() + f.close() + self._filePath = file + self._fileMtime = os.path.getmtime(file) + elif hasattr(file, 'read'): + source = file.read() # Can't set filename or mtime--they're not accessible. + elif file: + raise TypeError("'file' argument must be a filename string or file-like object") + + if self._filePath: + self._fileDirName, self._fileBaseName = os.path.split(self._filePath) + self._fileBaseNameRoot, self._fileBaseNameExt = os.path.splitext(self._fileBaseName) + + if not isinstance(source, basestring): + source = unicode(source) + # by converting to string here we allow objects such as other Templates + # to be passed in + + # Handle the #indent directive by converting it to other directives. + # (Over the long term we'll make it a real directive.) + if source == "": + warnings.warn("You supplied an empty string for the source!", ) + + else: + unicodeMatch = unicodeDirectiveRE.search(source) + encodingMatch = encodingDirectiveRE.search(source) + if unicodeMatch: + if encodingMatch: + raise ParseError( + self, "#encoding and #unicode are mutually exclusive! " + "Use one or the other.") + source = unicodeDirectiveRE.sub('', source) + if isinstance(source, str): + encoding = unicodeMatch.group(1) or 'ascii' + source = unicode(source, encoding) + elif encodingMatch: + encodings = encodingMatch.groups() + if len(encodings): + encoding = encodings[0] + source = source.decode(encoding) + else: + source = unicode(source) + + if source.find('#indent') != -1: #@@TR: undocumented hack + source = indentize(source) + + self._parser = self.parserClass(source, filename=self._filePath, compiler=self) + self._setupCompilerState() + + def __getattr__(self, name): + """Provide one-way access to the methods and attributes of the + ClassCompiler, and thereby the MethodCompilers as well. + + WARNING: Use .setMethods to assign the attributes of the ClassCompiler + from the methods of this class!!! or you will be assigning to attributes + of this object instead. + """ + if name in self.__dict__: + return self.__dict__[name] + elif hasattr(self.__class__, name): + return getattr(self.__class__, name) + elif self._activeClassesList and hasattr(self._activeClassesList[-1], name): + return getattr(self._activeClassesList[-1], name) + else: + raise AttributeError(name) + + def _initializeSettings(self): + self.updateSettings(copy.deepcopy(DEFAULT_COMPILER_SETTINGS)) + + def _setupCompilerState(self): + self._activeClassesList = [] + self._finishedClassesList = [] # listed by ordered + self._finishedClassIndex = {} # listed by name + self._moduleDef = None + self._moduleShBang = '#!/usr/bin/env python' + self._moduleEncoding = 'ascii' + self._moduleEncodingStr = '' + self._moduleHeaderLines = [] + self._moduleDocStringLines = [] + self._specialVars = {} + self._importStatements = [ + "import sys", + "import os", + "import os.path", + 'try:', + ' import builtins as builtin', + 'except ImportError:', + ' import __builtin__ as builtin', + "from os.path import getmtime, exists", + "import time", + "import types", + "from Cheetah.Version import MinCompatibleVersion as RequiredCheetahVersion", + "from Cheetah.Version import MinCompatibleVersionTuple as RequiredCheetahVersionTuple", + "from Cheetah.Template import Template", + "from Cheetah.DummyTransaction import *", + "from Cheetah.NameMapper import NotFound, valueForName, valueFromSearchList, valueFromFrameOrSearchList", + "from Cheetah.CacheRegion import CacheRegion", + "import Cheetah.Filters as Filters", + "import Cheetah.ErrorCatchers as ErrorCatchers", + ] + + self._importedVarNames = ['sys', + 'os', + 'os.path', + 'time', + 'types', + 'Template', + 'DummyTransaction', + 'NotFound', + 'Filters', + 'ErrorCatchers', + 'CacheRegion', + ] + + self._moduleConstants = [ + "VFFSL=valueFromFrameOrSearchList", + "VFSL=valueFromSearchList", + "VFN=valueForName", + "currentTime=time.time", + ] + + def compile(self): + classCompiler = self._spawnClassCompiler(self._mainClassName) + if self._baseclassName: + classCompiler.setBaseClass(self._baseclassName) + self._addActiveClassCompiler(classCompiler) + self._parser.parse() + self._swallowClassCompiler(self._popActiveClassCompiler()) + self._compiled = True + self._parser.cleanup() + + def _spawnClassCompiler(self, className, klass=None): + if klass is None: + klass = self.classCompilerClass + classCompiler = klass(className, + moduleCompiler=self, + mainMethodName=self.setting('mainMethodName'), + fileName=self._filePath, + settingsManager=self, + ) + return classCompiler + + def _addActiveClassCompiler(self, classCompiler): + self._activeClassesList.append(classCompiler) + + def _getActiveClassCompiler(self): + return self._activeClassesList[-1] + + def _popActiveClassCompiler(self): + return self._activeClassesList.pop() + + def _swallowClassCompiler(self, classCompiler): + classCompiler.cleanupState() + self._finishedClassesList.append( classCompiler ) + self._finishedClassIndex[classCompiler.className()] = classCompiler + return classCompiler + + def _finishedClasses(self): + return self._finishedClassesList + + def importedVarNames(self): + return self._importedVarNames + + def addImportedVarNames(self, varNames, raw_statement=None): + settings = self.settings() + if not varNames: + return + if not settings.get('useLegacyImportMode'): + if raw_statement and getattr(self, '_methodBodyChunks'): + self.addChunk(raw_statement) + else: + self._importedVarNames.extend(varNames) + + ## methods for adding stuff to the module and class definitions + + def setBaseClass(self, baseClassName): + if self._mainMethodNameArg: + self.setMainMethodName(self._mainMethodNameArg) + else: + self.setMainMethodName(self.setting('mainMethodNameForSubclasses')) + + if self.setting('handlerForExtendsDirective'): + handler = self.setting('handlerForExtendsDirective') + baseClassName = handler(compiler=self, baseClassName=baseClassName) + self._getActiveClassCompiler().setBaseClass(baseClassName) + elif (not self.setting('autoImportForExtendsDirective') + or baseClassName=='object' or baseClassName in self.importedVarNames()): + self._getActiveClassCompiler().setBaseClass(baseClassName) + # no need to import + else: + ################################################## + ## If the #extends directive contains a classname or modulename that isn't + # in self.importedVarNames() already, we assume that we need to add + # an implied 'from ModName import ClassName' where ModName == ClassName. + # - This is the case in WebKit servlet modules. + # - We also assume that the final . separates the classname from the + # module name. This might break if people do something really fancy + # with their dots and namespaces. + baseclasses = baseClassName.split(',') + for klass in baseclasses: + chunks = klass.split('.') + if len(chunks)==1: + self._getActiveClassCompiler().setBaseClass(klass) + if klass not in self.importedVarNames(): + modName = klass + # we assume the class name to be the module name + # and that it's not a builtin: + importStatement = "from %s import %s" % (modName, klass) + self.addImportStatement(importStatement) + self.addImportedVarNames((klass,)) + else: + needToAddImport = True + modName = chunks[0] + #print chunks, ':', self.importedVarNames() + for chunk in chunks[1:-1]: + if modName in self.importedVarNames(): + needToAddImport = False + finalBaseClassName = klass.replace(modName+'.', '') + self._getActiveClassCompiler().setBaseClass(finalBaseClassName) + break + else: + modName += '.'+chunk + if needToAddImport: + modName, finalClassName = '.'.join(chunks[:-1]), chunks[-1] + #if finalClassName != chunks[:-1][-1]: + if finalClassName != chunks[-2]: + # we assume the class name to be the module name + modName = '.'.join(chunks) + self._getActiveClassCompiler().setBaseClass(finalClassName) + importStatement = "from %s import %s" % (modName, finalClassName) + self.addImportStatement(importStatement) + self.addImportedVarNames( [finalClassName,] ) + + def setCompilerSetting(self, key, valueExpr): + self.setSetting(key, eval(valueExpr) ) + self._parser.configureParser() + + def setCompilerSettings(self, keywords, settingsStr): + KWs = keywords + merge = True + if 'nomerge' in KWs: + merge = False + + if 'reset' in KWs: + # @@TR: this is actually caught by the parser at the moment. + # subject to change in the future + self._initializeSettings() + self._parser.configureParser() + return + elif 'python' in KWs: + settingsReader = self.updateSettingsFromPySrcStr + # this comes from SettingsManager + else: + # this comes from SettingsManager + settingsReader = self.updateSettingsFromConfigStr + + settingsReader(settingsStr) + self._parser.configureParser() + + def setShBang(self, shBang): + self._moduleShBang = shBang + + def setModuleEncoding(self, encoding): + self._moduleEncoding = encoding + + def getModuleEncoding(self): + return self._moduleEncoding + + def addModuleHeader(self, line): + """Adds a header comment to the top of the generated module. + """ + self._moduleHeaderLines.append(line) + + def addModuleDocString(self, line): + """Adds a line to the generated module docstring. + """ + self._moduleDocStringLines.append(line) + + def addModuleGlobal(self, line): + """Adds a line of global module code. It is inserted after the import + statements and Cheetah default module constants. + """ + self._moduleConstants.append(line) + + def addSpecialVar(self, basename, contents, includeUnderscores=True): + """Adds module __specialConstant__ to the module globals. + """ + name = includeUnderscores and '__'+basename+'__' or basename + self._specialVars[name] = contents.strip() + + def addImportStatement(self, impStatement): + settings = self.settings() + if not self._methodBodyChunks or settings.get('useLegacyImportMode'): + # In the case where we are importing inline in the middle of a source block + # we don't want to inadvertantly import the module at the top of the file either + self._importStatements.append(impStatement) + + #@@TR 2005-01-01: there's almost certainly a cleaner way to do this! + importVarNames = impStatement[impStatement.find('import') + len('import'):].split(',') + importVarNames = [var.split()[-1] for var in importVarNames] # handles aliases + importVarNames = [var for var in importVarNames if not var == '*'] + self.addImportedVarNames(importVarNames, raw_statement=impStatement) #used by #extend for auto-imports + + def addAttribute(self, attribName, expr): + self._getActiveClassCompiler().addAttribute(attribName + ' =' + expr) + + def addComment(self, comm): + if re.match(r'#+$', comm): # skip bar comments + return + + specialVarMatch = specialVarRE.match(comm) + if specialVarMatch: + # @@TR: this is a bit hackish and is being replaced with + # #set module varName = ... + return self.addSpecialVar(specialVarMatch.group(1), + comm[specialVarMatch.end():]) + elif comm.startswith('doc:'): + addLine = self.addMethDocString + comm = comm[len('doc:'):].strip() + elif comm.startswith('doc-method:'): + addLine = self.addMethDocString + comm = comm[len('doc-method:'):].strip() + elif comm.startswith('doc-module:'): + addLine = self.addModuleDocString + comm = comm[len('doc-module:'):].strip() + elif comm.startswith('doc-class:'): + addLine = self.addClassDocString + comm = comm[len('doc-class:'):].strip() + elif comm.startswith('header:'): + addLine = self.addModuleHeader + comm = comm[len('header:'):].strip() + else: + addLine = self.addMethComment + + for line in comm.splitlines(): + addLine(line) + + ## methods for module code wrapping + + def getModuleCode(self): + if not self._compiled: + self.compile() + if self._moduleDef: + return self._moduleDef + else: + return self.wrapModuleDef() + + __str__ = getModuleCode + + def wrapModuleDef(self): + self.addSpecialVar('CHEETAH_docstring', self.setting('defDocStrMsg')) + self.addModuleGlobal('__CHEETAH_version__ = %r'%Version) + self.addModuleGlobal('__CHEETAH_versionTuple__ = %r'%(VersionTuple,)) + if self.setting('addTimestampsToCompilerOutput'): + self.addModuleGlobal('__CHEETAH_genTime__ = %r'%time.time()) + self.addModuleGlobal('__CHEETAH_genTimestamp__ = %r'%self.timestamp()) + if self._filePath: + timestamp = self.timestamp(self._fileMtime) + self.addModuleGlobal('__CHEETAH_src__ = %r'%self._filePath) + self.addModuleGlobal('__CHEETAH_srcLastModified__ = %r'%timestamp) + else: + self.addModuleGlobal('__CHEETAH_src__ = None') + self.addModuleGlobal('__CHEETAH_srcLastModified__ = None') + + moduleDef = """%(header)s +%(docstring)s + +################################################## +## DEPENDENCIES +%(imports)s + +################################################## +## MODULE CONSTANTS +%(constants)s +%(specialVars)s + +if __CHEETAH_versionTuple__ < RequiredCheetahVersionTuple: + raise AssertionError( + 'This template was compiled with Cheetah version' + ' %%s. Templates compiled before version %%s must be recompiled.'%%( + __CHEETAH_version__, RequiredCheetahVersion)) + +################################################## +## CLASSES + +%(classes)s + +## END CLASS DEFINITION + +if not hasattr(%(mainClassName)s, '_initCheetahAttributes'): + templateAPIClass = getattr(%(mainClassName)s, '_CHEETAH_templateClass', Template) + templateAPIClass._addCheetahPlumbingCodeToClass(%(mainClassName)s) + +%(footer)s +""" % {'header': self.moduleHeader(), + 'docstring': self.moduleDocstring(), + 'specialVars': self.specialVars(), + 'imports': self.importStatements(), + 'constants': self.moduleConstants(), + 'classes': self.classDefs(), + 'footer': self.moduleFooter(), + 'mainClassName': self._mainClassName, + } + + self._moduleDef = moduleDef + return moduleDef + + def timestamp(self, theTime=None): + if not theTime: + theTime = time.time() + return time.asctime(time.localtime(theTime)) + + def moduleHeader(self): + header = self._moduleShBang + '\n' + header += self._moduleEncodingStr + '\n' + if self._moduleHeaderLines: + offSet = self.setting('commentOffset') + + header += ( + '#' + ' '*offSet + + ('\n#'+ ' '*offSet).join(self._moduleHeaderLines) + '\n') + + return header + + def moduleDocstring(self): + if not self._moduleDocStringLines: + return '' + + return ('"""' + + '\n'.join(self._moduleDocStringLines) + + '\n"""\n') + + def specialVars(self): + chunks = [] + theVars = self._specialVars + keys = sorted(theVars.keys()) + for key in keys: + chunks.append(key + ' = ' + repr(theVars[key]) ) + return '\n'.join(chunks) + + def importStatements(self): + return '\n'.join(self._importStatements) + + def moduleConstants(self): + return '\n'.join(self._moduleConstants) + + def classDefs(self): + classDefs = [klass.classDef() for klass in self._finishedClasses()] + return '\n\n'.join(classDefs) + + def moduleFooter(self): + return """ +# CHEETAH was developed by Tavis Rudd and Mike Orr +# with code, advice and input from many other volunteers. +# For more information visit http://www.CheetahTemplate.org/ + +################################################## +## if run from command line: +if __name__ == '__main__': + from Cheetah.TemplateCmdLineIface import CmdLineIface + CmdLineIface(templateObj=%(className)s()).run() + +""" % {'className':self._mainClassName} + + +################################################## +## Make Compiler an alias for ModuleCompiler + +Compiler = ModuleCompiler === added file 'cheetah/DirectiveAnalyzer.py' --- cheetah/DirectiveAnalyzer.py 1970-01-01 00:00:00 +0000 +++ cheetah/DirectiveAnalyzer.py 2010-10-19 19:11:10 +0000 @@ -0,0 +1,98 @@ +#!/usr/bin/env python + +import os +import pprint + +try: + from functools import reduce +except ImportError: + # Assume we have reduce + pass + +from Cheetah import Parser +from Cheetah import Compiler +from Cheetah import Template + +class Analyzer(Parser.Parser): + def __init__(self, *args, **kwargs): + self.calls = {} + super(Analyzer, self).__init__(*args, **kwargs) + + def eatDirective(self): + directive = self.matchDirective() + try: + self.calls[directive] += 1 + except KeyError: + self.calls[directive] = 1 + super(Analyzer, self).eatDirective() + +class AnalysisCompiler(Compiler.ModuleCompiler): + parserClass = Analyzer + + +def analyze(source): + klass = Template.Template.compile(source, compilerClass=AnalysisCompiler) + return klass._CHEETAH_compilerInstance._parser.calls + +def main_file(f): + fd = open(f, 'r') + try: + print u'>>> Analyzing %s' % f + calls = analyze(fd.read()) + return calls + finally: + fd.close() + + +def _find_templates(directory, suffix): + for root, dirs, files in os.walk(directory): + for f in files: + if not f.endswith(suffix): + continue + yield root + os.path.sep + f + +def _analyze_templates(iterable): + for template in iterable: + yield main_file(template) + +def main_dir(opts): + results = _analyze_templates(_find_templates(opts.dir, opts.suffix)) + totals = {} + for series in results: + if not series: + continue + for k, v in series.iteritems(): + try: + totals[k] += v + except KeyError: + totals[k] = v + return totals + + +def main(): + from optparse import OptionParser + op = OptionParser() + op.add_option('-f', '--file', dest='file', default=None, + help='Specify a single file to analyze') + op.add_option('-d', '--dir', dest='dir', default=None, + help='Specify a directory of templates to analyze') + op.add_option('--suffix', default='tmpl', dest='suffix', + help='Specify a custom template file suffix for the -d option (default: "tmpl")') + opts, args = op.parse_args() + + if not opts.file and not opts.dir: + op.print_help() + return + + results = None + if opts.file: + results = main_file(opts.file) + if opts.dir: + results = main_dir(opts) + + pprint.pprint(results) + + +if __name__ == '__main__': + main() + === added file 'cheetah/Django.py' --- cheetah/Django.py 1970-01-01 00:00:00 +0000 +++ cheetah/Django.py 2010-10-19 19:11:10 +0000 @@ -0,0 +1,16 @@ +import Cheetah.Template + +def render(template_file, **kwargs): + ''' + Cheetah.Django.render() takes the template filename + (the filename should be a file in your Django + TEMPLATE_DIRS) + + Any additional keyword arguments are passed into the + template are propogated into the template's searchList + ''' + import django.http + import django.template.loader + source, loader = django.template.loader.find_template_source(template_file) + t = Cheetah.Template.Template(source, searchList=[kwargs]) + return django.http.HttpResponse(t.__str__()) === added file 'cheetah/DummyTransaction.py' --- cheetah/DummyTransaction.py 1970-01-01 00:00:00 +0000 +++ cheetah/DummyTransaction.py 2010-10-19 19:11:10 +0000 @@ -0,0 +1,108 @@ + +''' +Provides dummy Transaction and Response classes is used by Cheetah in place +of real Webware transactions when the Template obj is not used directly as a +Webware servlet. + +Warning: This may be deprecated in the future, please do not rely on any +specific DummyTransaction or DummyResponse behavior +''' + +import logging +import types + +class DummyResponseFailure(Exception): + pass + +class DummyResponse(object): + ''' + A dummy Response class is used by Cheetah in place of real Webware + Response objects when the Template obj is not used directly as a Webware + servlet + ''' + def __init__(self): + self._outputChunks = [] + + def flush(self): + pass + + def safeConvert(self, chunk): + # Exceptionally gross, but the safest way + # I've found to ensure I get a legit unicode object + if not chunk: + return u'' + if isinstance(chunk, unicode): + return chunk + try: + return chunk.decode('utf-8', 'strict') + except UnicodeDecodeError: + try: + return chunk.decode('latin-1', 'strict') + except UnicodeDecodeError: + return chunk.decode('ascii', 'ignore') + except AttributeError: + return unicode(chunk, errors='ignore') + return chunk + + def write(self, value): + self._outputChunks.append(value) + + def writeln(self, txt): + write(txt) + write('\n') + + def getvalue(self, outputChunks=None): + chunks = outputChunks or self._outputChunks + try: + return u''.join(chunks) + except UnicodeDecodeError, ex: + logging.debug('Trying to work around a UnicodeDecodeError in getvalue()') + logging.debug('...perhaps you could fix "%s" while you\'re debugging') + return ''.join((self.safeConvert(c) for c in chunks)) + + def writelines(self, *lines): + ## not used + [self.writeln(ln) for ln in lines] + + +class DummyTransaction(object): + ''' + A dummy Transaction class is used by Cheetah in place of real Webware + transactions when the Template obj is not used directly as a Webware + servlet. + + It only provides a response object and method. All other methods and + attributes make no sense in this context. + ''' + def __init__(self, *args, **kwargs): + self._response = None + + def response(self, resp=None): + if self._response is None: + self._response = resp or DummyResponse() + return self._response + + +class TransformerResponse(DummyResponse): + def __init__(self, *args, **kwargs): + super(TransformerResponse, self).__init__(*args, **kwargs) + self._filter = None + + def getvalue(self, **kwargs): + output = super(TransformerResponse, self).getvalue(**kwargs) + if self._filter: + _filter = self._filter + if isinstance(_filter, type): + _filter = _filter() + return _filter.filter(output) + return output + + +class TransformerTransaction(object): + def __init__(self, *args, **kwargs): + self._response = None + def response(self): + if self._response: + return self._response + return TransformerResponse() + === added file 'cheetah/ErrorCatchers.py' --- cheetah/ErrorCatchers.py 1970-01-01 00:00:00 +0000 +++ cheetah/ErrorCatchers.py 2010-10-19 19:11:10 +0000 @@ -0,0 +1,62 @@ +# $Id: ErrorCatchers.py,v 1.7 2005/01/03 19:59:07 tavis_rudd Exp $ +"""ErrorCatcher class for Cheetah Templates + +Meta-Data +================================================================================ +Author: Tavis Rudd +Version: $Revision: 1.7 $ +Start Date: 2001/08/01 +Last Revision Date: $Date: 2005/01/03 19:59:07 $ +""" +__author__ = "Tavis Rudd " +__revision__ = "$Revision: 1.7 $"[11:-2] + +import time +from Cheetah.NameMapper import NotFound + +class Error(Exception): + pass + +class ErrorCatcher: + _exceptionsToCatch = (NotFound,) + + def __init__(self, templateObj): + pass + + def exceptions(self): + return self._exceptionsToCatch + + def warn(self, exc_val, code, rawCode, lineCol): + return rawCode +## make an alias +Echo = ErrorCatcher + +class BigEcho(ErrorCatcher): + def warn(self, exc_val, code, rawCode, lineCol): + return "="*15 + "<" + rawCode + " could not be found>" + "="*15 + +class KeyError(ErrorCatcher): + def warn(self, exc_val, code, rawCode, lineCol): + raise KeyError("no '%s' in this Template Object's Search List" % rawCode) + +class ListErrors(ErrorCatcher): + """Accumulate a list of errors.""" + _timeFormat = "%c" + + def __init__(self, templateObj): + ErrorCatcher.__init__(self, templateObj) + self._errors = [] + + def warn(self, exc_val, code, rawCode, lineCol): + dict = locals().copy() + del dict['self'] + dict['time'] = time.strftime(self._timeFormat, + time.localtime(time.time())) + self._errors.append(dict) + return rawCode + + def listErrors(self): + """Return the list of errors.""" + return self._errors + + === added file 'cheetah/FileUtils.py' --- cheetah/FileUtils.py 1970-01-01 00:00:00 +0000 +++ cheetah/FileUtils.py 2010-10-19 19:11:10 +0000 @@ -0,0 +1,357 @@ + +from glob import glob +import os +from os import listdir +import os.path +import re +from tempfile import mktemp + +def _escapeRegexChars(txt, + escapeRE=re.compile(r'([\$\^\*\+\.\?\{\}\[\]\(\)\|\\])')): + return escapeRE.sub(r'\\\1', txt) + +def findFiles(*args, **kw): + """Recursively find all the files matching a glob pattern. + + This function is a wrapper around the FileFinder class. See its docstring + for details about the accepted arguments, etc.""" + + return FileFinder(*args, **kw).files() + +def replaceStrInFiles(files, theStr, repl): + + """Replace all instances of 'theStr' with 'repl' for each file in the 'files' + list. Returns a dictionary with data about the matches found. + + This is like string.replace() on a multi-file basis. + + This function is a wrapper around the FindAndReplace class. See its + docstring for more details.""" + + pattern = _escapeRegexChars(theStr) + return FindAndReplace(files, pattern, repl).results() + +def replaceRegexInFiles(files, pattern, repl): + + """Replace all instances of regex 'pattern' with 'repl' for each file in the + 'files' list. Returns a dictionary with data about the matches found. + + This is like re.sub on a multi-file basis. + + This function is a wrapper around the FindAndReplace class. See its + docstring for more details.""" + + return FindAndReplace(files, pattern, repl).results() + + +################################################## +## CLASSES + +class FileFinder: + + """Traverses a directory tree and finds all files in it that match one of + the specified glob patterns.""" + + def __init__(self, rootPath, + globPatterns=('*',), + ignoreBasenames=('CVS', '.svn'), + ignoreDirs=(), + ): + + self._rootPath = rootPath + self._globPatterns = globPatterns + self._ignoreBasenames = ignoreBasenames + self._ignoreDirs = ignoreDirs + self._files = [] + + self.walkDirTree(rootPath) + + def walkDirTree(self, dir='.', + + listdir=os.listdir, + isdir=os.path.isdir, + join=os.path.join, + ): + + """Recursively walk through a directory tree and find matching files.""" + processDir = self.processDir + filterDir = self.filterDir + + pendingDirs = [dir] + addDir = pendingDirs.append + getDir = pendingDirs.pop + + while pendingDirs: + dir = getDir() + ## process this dir + processDir(dir) + + ## and add sub-dirs + for baseName in listdir(dir): + fullPath = join(dir, baseName) + if isdir(fullPath): + if filterDir(baseName, fullPath): + addDir( fullPath ) + + def filterDir(self, baseName, fullPath): + + """A hook for filtering out certain dirs. """ + + return not (baseName in self._ignoreBasenames or + fullPath in self._ignoreDirs) + + def processDir(self, dir, glob=glob): + extend = self._files.extend + for pattern in self._globPatterns: + extend( glob(os.path.join(dir, pattern)) ) + + def files(self): + return self._files + +class _GenSubberFunc: + + """Converts a 'sub' string in the form that one feeds to re.sub (backrefs, + groups, etc.) into a function that can be used to do the substitutions in + the FindAndReplace class.""" + + backrefRE = re.compile(r'\\([1-9][0-9]*)') + groupRE = re.compile(r'\\g<([a-zA-Z_][a-zA-Z_]*)>') + + def __init__(self, replaceStr): + self._src = replaceStr + self._pos = 0 + self._codeChunks = [] + self.parse() + + def src(self): + return self._src + + def pos(self): + return self._pos + + def setPos(self, pos): + self._pos = pos + + def atEnd(self): + return self._pos >= len(self._src) + + def advance(self, offset=1): + self._pos += offset + + def readTo(self, to, start=None): + if start == None: + start = self._pos + self._pos = to + if self.atEnd(): + return self._src[start:] + else: + return self._src[start:to] + + ## match and get methods + + def matchBackref(self): + return self.backrefRE.match(self.src(), self.pos()) + + def getBackref(self): + m = self.matchBackref() + self.setPos(m.end()) + return m.group(1) + + def matchGroup(self): + return self.groupRE.match(self.src(), self.pos()) + + def getGroup(self): + m = self.matchGroup() + self.setPos(m.end()) + return m.group(1) + + ## main parse loop and the eat methods + + def parse(self): + while not self.atEnd(): + if self.matchBackref(): + self.eatBackref() + elif self.matchGroup(): + self.eatGroup() + else: + self.eatStrConst() + + def eatStrConst(self): + startPos = self.pos() + while not self.atEnd(): + if self.matchBackref() or self.matchGroup(): + break + else: + self.advance() + strConst = self.readTo(self.pos(), start=startPos) + self.addChunk(repr(strConst)) + + def eatBackref(self): + self.addChunk( 'm.group(' + self.getBackref() + ')' ) + + def eatGroup(self): + self.addChunk( 'm.group("' + self.getGroup() + '")' ) + + def addChunk(self, chunk): + self._codeChunks.append(chunk) + + ## code wrapping methods + + def codeBody(self): + return ', '.join(self._codeChunks) + + def code(self): + return "def subber(m):\n\treturn ''.join([%s])\n" % (self.codeBody()) + + def subberFunc(self): + exec(self.code()) + return subber + + +class FindAndReplace: + + """Find and replace all instances of 'patternOrRE' with 'replacement' for + each file in the 'files' list. This is a multi-file version of re.sub(). + + 'patternOrRE' can be a raw regex pattern or + a regex object as generated by the re module. 'replacement' can be any + string that would work with patternOrRE.sub(replacement, fileContents). + """ + + def __init__(self, files, patternOrRE, replacement, + recordResults=True): + + + if isinstance(patternOrRE, basestring): + self._regex = re.compile(patternOrRE) + else: + self._regex = patternOrRE + if isinstance(replacement, basestring): + self._subber = _GenSubberFunc(replacement).subberFunc() + else: + self._subber = replacement + + self._pattern = pattern = self._regex.pattern + self._files = files + self._results = {} + self._recordResults = recordResults + + ## see if we should use pgrep to do the file matching + self._usePgrep = False + if (os.popen3('pgrep')[2].read()).startswith('Usage:'): + ## now check to make sure pgrep understands the pattern + tmpFile = mktemp() + open(tmpFile, 'w').write('#') + if not (os.popen3('pgrep "' + pattern + '" ' + tmpFile)[2].read()): + # it didn't print an error msg so we're ok + self._usePgrep = True + os.remove(tmpFile) + + self._run() + + def results(self): + return self._results + + def _run(self): + regex = self._regex + subber = self._subDispatcher + usePgrep = self._usePgrep + pattern = self._pattern + for file in self._files: + if not os.path.isfile(file): + continue # skip dirs etc. + + self._currFile = file + found = False + if 'orig' in locals(): + del orig + if self._usePgrep: + if os.popen('pgrep "' + pattern + '" ' + file ).read(): + found = True + else: + orig = open(file).read() + if regex.search(orig): + found = True + if found: + if 'orig' not in locals(): + orig = open(file).read() + new = regex.sub(subber, orig) + open(file, 'w').write(new) + + def _subDispatcher(self, match): + if self._recordResults: + if self._currFile not in self._results: + res = self._results[self._currFile] = {} + res['count'] = 0 + res['matches'] = [] + else: + res = self._results[self._currFile] + res['count'] += 1 + res['matches'].append({'contents': match.group(), + 'start': match.start(), + 'end': match.end(), + } + ) + return self._subber(match) + + +class SourceFileStats: + + """ + """ + + _fileStats = None + + def __init__(self, files): + self._fileStats = stats = {} + for file in files: + stats[file] = self.getFileStats(file) + + def rawStats(self): + return self._fileStats + + def summary(self): + codeLines = 0 + blankLines = 0 + commentLines = 0 + totalLines = 0 + for fileStats in self.rawStats().values(): + codeLines += fileStats['codeLines'] + blankLines += fileStats['blankLines'] + commentLines += fileStats['commentLines'] + totalLines += fileStats['totalLines'] + + stats = {'codeLines': codeLines, + 'blankLines': blankLines, + 'commentLines': commentLines, + 'totalLines': totalLines, + } + return stats + + def printStats(self): + pass + + def getFileStats(self, fileName): + codeLines = 0 + blankLines = 0 + commentLines = 0 + commentLineRe = re.compile(r'\s#.*$') + blankLineRe = re.compile('\s$') + lines = open(fileName).read().splitlines() + totalLines = len(lines) + + for line in lines: + if commentLineRe.match(line): + commentLines += 1 + elif blankLineRe.match(line): + blankLines += 1 + else: + codeLines += 1 + + stats = {'codeLines': codeLines, + 'blankLines': blankLines, + 'commentLines': commentLines, + 'totalLines': totalLines, + } + + return stats === added file 'cheetah/Filters.py' --- cheetah/Filters.py 1970-01-01 00:00:00 +0000 +++ cheetah/Filters.py 2010-10-19 19:11:10 +0000 @@ -0,0 +1,212 @@ +''' + Filters for the #filter directive as well as #transform + + #filter results in output filters Cheetah's $placeholders . + #transform results in a filter on the entirety of the output +''' +import sys + +# Additional entities WebSafe knows how to transform. No need to include +# '<', '>' or '&' since those will have been done already. +webSafeEntities = {' ': ' ', '"': '"'} + +class Filter(object): + """A baseclass for the Cheetah Filters.""" + + def __init__(self, template=None): + """Setup a reference to the template that is using the filter instance. + This reference isn't used by any of the standard filters, but is + available to Filter subclasses, should they need it. + + Subclasses should call this method. + """ + self.template = template + + def filter(self, val, encoding=None, str=str, **kw): + ''' + Pass Unicode strings through unmolested, unless an encoding is specified. + ''' + if val is None: + return u'' + if isinstance(val, unicode): + # ignore the encoding and return the unicode object + return val + else: + try: + return unicode(val) + except UnicodeDecodeError: + # we could put more fallbacks here, but we'll just pass the str + # on and let DummyTransaction worry about it + return str(val) + +RawOrEncodedUnicode = Filter + +EncodeUnicode = Filter + +class Markdown(EncodeUnicode): + ''' + Markdown will change regular strings to Markdown + (http://daringfireball.net/projects/markdown/) + + Such that: + My Header + ========= + Becaomes: +

    My Header

    + + and so on. + + Markdown is meant to be used with the #transform + tag, as it's usefulness with #filter is marginal at + best + ''' + def filter(self, value, **kwargs): + # This is a bit of a hack to allow outright embedding of the markdown module + try: + import markdown + except ImportError: + print('>>> Exception raised importing the "markdown" module') + print('>>> Are you sure you have the ElementTree module installed?') + print(' http://effbot.org/downloads/#elementtree') + raise + + encoded = super(Markdown, self).filter(value, **kwargs) + return markdown.markdown(encoded) + +class CodeHighlighter(EncodeUnicode): + ''' + The CodeHighlighter filter depends on the "pygments" module which you can + download and install from: http://pygments.org + + What the CodeHighlighter assumes the string that it's receiving is source + code and uses pygments.lexers.guess_lexer() to try to guess which parser + to use when highlighting it. + + CodeHighlighter will return the HTML and CSS to render the code block, syntax + highlighted, in a browser + + NOTE: I had an issue installing pygments on Linux/amd64/Python 2.6 dealing with + importing of pygments.lexers, I was able to correct the failure by adding: + raise ImportError + to line 39 of pygments/plugin.py (since importing pkg_resources was causing issues) + ''' + def filter(self, source, **kwargs): + encoded = super(CodeHighlighter, self).filter(source, **kwargs) + try: + from pygments import highlight + from pygments import lexers + from pygments import formatters + except ImportError, ex: + print('<%s> - Failed to import pygments! (%s)' % (self.__class__.__name__, ex)) + print('-- You may need to install it from: http://pygments.org') + return encoded + + lexer = None + try: + lexer = lexers.guess_lexer(source) + except lexers.ClassNotFound: + lexer = lexers.PythonLexer() + + formatter = formatters.HtmlFormatter(cssclass='code_highlighter') + encoded = highlight(encoded, lexer, formatter) + css = formatter.get_style_defs('.code_highlighter') + return '''%(source)s''' % {'css' : css, 'source' : encoded} + + + +class MaxLen(Filter): + def filter(self, val, **kw): + """Replace None with '' and cut off at maxlen.""" + + output = super(MaxLen, self).filter(val, **kw) + if 'maxlen' in kw and len(output) > kw['maxlen']: + return output[:kw['maxlen']] + return output + +class WebSafe(Filter): + """Escape HTML entities in $placeholders. + """ + def filter(self, val, **kw): + s = super(WebSafe, self).filter(val, **kw) + # These substitutions are copied from cgi.escape(). + s = s.replace("&", "&") # Must be done first! + s = s.replace("<", "<") + s = s.replace(">", ">") + # Process the additional transformations if any. + if 'also' in kw: + also = kw['also'] + entities = webSafeEntities # Global variable. + for k in also: + if k in entities: + v = entities[k] + else: + v = "&#%s;" % ord(k) + s = s.replace(k, v) + return s + + +class Strip(Filter): + """Strip leading/trailing whitespace but preserve newlines. + + This filter goes through the value line by line, removing leading and + trailing whitespace on each line. It does not strip newlines, so every + input line corresponds to one output line, with its trailing newline intact. + + We do not use val.split('\n') because that would squeeze out consecutive + blank lines. Instead, we search for each newline individually. This + makes us unable to use the fast C .split method, but it makes the filter + much more widely useful. + + This filter is intended to be usable both with the #filter directive and + with the proposed #sed directive (which has not been ratified yet.) + """ + def filter(self, val, **kw): + s = super(Strip, self).filter(val, **kw) + result = [] + start = 0 # The current line will be s[start:end]. + while True: # Loop through each line. + end = s.find('\n', start) # Find next newline. + if end == -1: # If no more newlines. + break + chunk = s[start:end].strip() + result.append(chunk) + result.append('\n') + start = end + 1 + # Write the unfinished portion after the last newline, if any. + chunk = s[start:].strip() + result.append(chunk) + return "".join(result) + +class StripSqueeze(Filter): + """Canonicalizes every chunk of whitespace to a single space. + + Strips leading/trailing whitespace. Removes all newlines, so multi-line + input is joined into one ling line with NO trailing newline. + """ + def filter(self, val, **kw): + s = super(StripSqueeze, self).filter(val, **kw) + s = s.split() + return " ".join(s) + +################################################## +## MAIN ROUTINE -- testing + +def test(): + s1 = "abc <=> &" + s2 = " asdf \n\t 1 2 3\n" + print("WebSafe INPUT:", repr(s1)) + print(" WebSafe:", repr(WebSafe().filter(s1))) + + print() + print(" Strip INPUT:", repr(s2)) + print(" Strip:", repr(Strip().filter(s2))) + print("StripSqueeze:", repr(StripSqueeze().filter(s2))) + + print("Unicode:", repr(EncodeUnicode().filter(u'aoeu12345\u1234'))) + +if __name__ == "__main__": + test() + +# vim: shiftwidth=4 tabstop=4 expandtab === added file 'cheetah/ImportHooks.py' --- cheetah/ImportHooks.py 1970-01-01 00:00:00 +0000 +++ cheetah/ImportHooks.py 2010-10-19 19:11:10 +0000 @@ -0,0 +1,129 @@ +#!/usr/bin/env python + +""" +Provides some import hooks to allow Cheetah's .tmpl files to be imported +directly like Python .py modules. + +To use these: + import Cheetah.ImportHooks + Cheetah.ImportHooks.install() +""" + +import sys +import os.path +import types +import __builtin__ +import imp +from threading import RLock +import string +import traceback +import types + +from Cheetah import ImportManager +from Cheetah.ImportManager import DirOwner +from Cheetah.Compiler import Compiler +from Cheetah.convertTmplPathToModuleName import convertTmplPathToModuleName + +_installed = False + +################################################## +## HELPER FUNCS + +_cacheDir = [] +def setCacheDir(cacheDir): + global _cacheDir + _cacheDir.append(cacheDir) + +################################################## +## CLASSES + +class CheetahDirOwner(DirOwner): + _lock = RLock() + _acquireLock = _lock.acquire + _releaseLock = _lock.release + + templateFileExtensions = ('.tmpl',) + + def getmod(self, name): + self._acquireLock() + try: + mod = DirOwner.getmod(self, name) + if mod: + return mod + + for ext in self.templateFileExtensions: + tmplPath = os.path.join(self.path, name + ext) + if os.path.exists(tmplPath): + try: + return self._compile(name, tmplPath) + except: + # @@TR: log the error + exc_txt = traceback.format_exc() + exc_txt =' '+(' \n'.join(exc_txt.splitlines())) + raise ImportError( + 'Error while compiling Cheetah module' + ' %(name)s, original traceback follows:\n%(exc_txt)s'%locals()) + ## + return None + + finally: + self._releaseLock() + + def _compile(self, name, tmplPath): + ## @@ consider adding an ImportError raiser here + code = str(Compiler(file=tmplPath, moduleName=name, + mainClassName=name)) + if _cacheDir: + __file__ = os.path.join(_cacheDir[0], + convertTmplPathToModuleName(tmplPath)) + '.py' + try: + open(__file__, 'w').write(code) + except OSError: + ## @@ TR: need to add some error code here + traceback.print_exc(file=sys.stderr) + __file__ = tmplPath + else: + __file__ = tmplPath + co = compile(code+'\n', __file__, 'exec') + + mod = types.ModuleType(name) + mod.__file__ = co.co_filename + if _cacheDir: + mod.__orig_file__ = tmplPath # @@TR: this is used in the WebKit + # filemonitoring code + mod.__co__ = co + return mod + + +################################################## +## FUNCTIONS + +def install(templateFileExtensions=('.tmpl',)): + """Install the Cheetah Import Hooks""" + + global _installed + if not _installed: + CheetahDirOwner.templateFileExtensions = templateFileExtensions + import __builtin__ + if isinstance(__builtin__.__import__, types.BuiltinFunctionType): + global __oldimport__ + __oldimport__ = __builtin__.__import__ + ImportManager._globalOwnerTypes.insert(0, CheetahDirOwner) + #ImportManager._globalOwnerTypes.append(CheetahDirOwner) + global _manager + _manager=ImportManager.ImportManager() + _manager.setThreaded() + _manager.install() + +def uninstall(): + """Uninstall the Cheetah Import Hooks""" + global _installed + if not _installed: + import __builtin__ + if isinstance(__builtin__.__import__, types.MethodType): + __builtin__.__import__ = __oldimport__ + global _manager + del _manager + +if __name__ == '__main__': + install() === added file 'cheetah/ImportManager.py' --- cheetah/ImportManager.py 1970-01-01 00:00:00 +0000 +++ cheetah/ImportManager.py 2010-10-19 19:11:10 +0000 @@ -0,0 +1,541 @@ +""" +Provides an emulator/replacement for Python's standard import system. + +@@TR: Be warned that Import Hooks are in the deepest, darkest corner of Python's +jungle. If you need to start hacking with this, be prepared to get lost for a +while. Also note, this module predates the newstyle import hooks in Python 2.3 +http://www.python.org/peps/pep-0302.html. + + +This is a hacked/documented version of Gordon McMillan's iu.py. I have: + + - made it a little less terse + + - added docstrings and explanatations + + - standardized the variable naming scheme + + - reorganized the code layout to enhance readability + +""" + +import sys +import imp +import marshal + +_installed = False + +# _globalOwnerTypes is defined at the bottom of this file + +_os_stat = _os_path_join = _os_getcwd = _os_path_dirname = None + +################################################## +## FUNCTIONS + +def _os_bootstrap(): + """Set up 'os' module replacement functions for use during import bootstrap.""" + + names = sys.builtin_module_names + + join = dirname = None + if 'posix' in names: + sep = '/' + from posix import stat, getcwd + elif 'nt' in names: + sep = '\\' + from nt import stat, getcwd + elif 'dos' in names: + sep = '\\' + from dos import stat, getcwd + elif 'os2' in names: + sep = '\\' + from os2 import stat, getcwd + elif 'mac' in names: + from mac import stat, getcwd + def join(a, b): + if a == '': + return b + if ':' not in a: + a = ':' + a + if a[-1:] != ':': + a = a + ':' + return a + b + else: + raise ImportError('no os specific module found') + + if join is None: + def join(a, b, sep=sep): + if a == '': + return b + lastchar = a[-1:] + if lastchar == '/' or lastchar == sep: + return a + b + return a + sep + b + + if dirname is None: + def dirname(a, sep=sep): + for i in range(len(a)-1, -1, -1): + c = a[i] + if c == '/' or c == sep: + return a[:i] + return '' + + global _os_stat + _os_stat = stat + + global _os_path_join + _os_path_join = join + + global _os_path_dirname + _os_path_dirname = dirname + + global _os_getcwd + _os_getcwd = getcwd + +_os_bootstrap() + +def packageName(s): + for i in range(len(s)-1, -1, -1): + if s[i] == '.': + break + else: + return '' + return s[:i] + +def nameSplit(s): + rslt = [] + i = j = 0 + for j in range(len(s)): + if s[j] == '.': + rslt.append(s[i:j]) + i = j+1 + if i < len(s): + rslt.append(s[i:]) + return rslt + +def getPathExt(fnm): + for i in range(len(fnm)-1, -1, -1): + if fnm[i] == '.': + return fnm[i:] + return '' + +def pathIsDir(pathname): + "Local replacement for os.path.isdir()." + try: + s = _os_stat(pathname) + except OSError: + return None + return (s[0] & 0170000) == 0040000 + +def getDescr(fnm): + ext = getPathExt(fnm) + for (suffix, mode, typ) in imp.get_suffixes(): + if suffix == ext: + return (suffix, mode, typ) + +################################################## +## CLASSES + +class Owner: + + """An Owner does imports from a particular piece of turf That is, there's + an Owner for each thing on sys.path There are owners for directories and + .pyz files. There could be owners for zip files, or even URLs. A + shadowpath (a dictionary mapping the names in sys.path to their owners) is + used so that sys.path (or a package's __path__) is still a bunch of strings, + """ + + def __init__(self, path): + self.path = path + + def __str__(self): + return self.path + + def getmod(self, nm): + return None + +class DirOwner(Owner): + + def __init__(self, path): + if path == '': + path = _os_getcwd() + if not pathIsDir(path): + raise ValueError("%s is not a directory" % path) + Owner.__init__(self, path) + + def getmod(self, nm, + getsuffixes=imp.get_suffixes, loadco=marshal.loads, newmod=imp.new_module): + + pth = _os_path_join(self.path, nm) + + possibles = [(pth, 0, None)] + if pathIsDir(pth): + possibles.insert(0, (_os_path_join(pth, '__init__'), 1, pth)) + py = pyc = None + for pth, ispkg, pkgpth in possibles: + for ext, mode, typ in getsuffixes(): + attempt = pth+ext + try: + st = _os_stat(attempt) + except: + pass + else: + if typ == imp.C_EXTENSION: + fp = open(attempt, 'rb') + mod = imp.load_module(nm, fp, attempt, (ext, mode, typ)) + mod.__file__ = attempt + return mod + elif typ == imp.PY_SOURCE: + py = (attempt, st) + else: + pyc = (attempt, st) + if py or pyc: + break + if py is None and pyc is None: + return None + while True: + if pyc is None or py and pyc[1][8] < py[1][8]: + try: + co = compile(open(py[0], 'r').read()+'\n', py[0], 'exec') + break + except SyntaxError, e: + print("Invalid syntax in %s" % py[0]) + print(e.args) + raise + elif pyc: + stuff = open(pyc[0], 'rb').read() + try: + co = loadco(stuff[8:]) + break + except (ValueError, EOFError): + pyc = None + else: + return None + mod = newmod(nm) + mod.__file__ = co.co_filename + if ispkg: + mod.__path__ = [pkgpth] + subimporter = PathImportDirector(mod.__path__) + mod.__importsub__ = subimporter.getmod + mod.__co__ = co + return mod + + +class ImportDirector(Owner): + """ImportDirectors live on the metapath There's one for builtins, one for + frozen modules, and one for sys.path Windows gets one for modules gotten + from the Registry Mac would have them for PY_RESOURCE modules etc. A + generalization of Owner - their concept of 'turf' is broader""" + + pass + +class BuiltinImportDirector(ImportDirector): + """Directs imports of builtin modules""" + def __init__(self): + self.path = 'Builtins' + + def getmod(self, nm, isbuiltin=imp.is_builtin): + if isbuiltin(nm): + mod = imp.load_module(nm, None, nm, ('', '', imp.C_BUILTIN)) + return mod + return None + +class FrozenImportDirector(ImportDirector): + """Directs imports of frozen modules""" + + def __init__(self): + self.path = 'FrozenModules' + + def getmod(self, nm, + isFrozen=imp.is_frozen, loadMod=imp.load_module): + if isFrozen(nm): + mod = loadMod(nm, None, nm, ('', '', imp.PY_FROZEN)) + if hasattr(mod, '__path__'): + mod.__importsub__ = lambda name, pname=nm, owner=self: owner.getmod(pname+'.'+name) + return mod + return None + + +class RegistryImportDirector(ImportDirector): + """Directs imports of modules stored in the Windows Registry""" + + def __init__(self): + self.path = "WindowsRegistry" + self.map = {} + try: + import win32api + ## import win32con + except ImportError: + pass + else: + HKEY_CURRENT_USER = -2147483647 + HKEY_LOCAL_MACHINE = -2147483646 + KEY_ALL_ACCESS = 983103 + subkey = r"Software\Python\PythonCore\%s\Modules" % sys.winver + for root in (HKEY_CURRENT_USER, HKEY_LOCAL_MACHINE): + try: + hkey = win32api.RegOpenKeyEx(root, subkey, 0, KEY_ALL_ACCESS) + except: + pass + else: + numsubkeys, numvalues, lastmodified = win32api.RegQueryInfoKey(hkey) + for i in range(numsubkeys): + subkeyname = win32api.RegEnumKey(hkey, i) + hskey = win32api.RegOpenKeyEx(hkey, subkeyname, 0, KEY_ALL_ACCESS) + val = win32api.RegQueryValueEx(hskey, '') + desc = getDescr(val[0]) + self.map[subkeyname] = (val[0], desc) + hskey.Close() + hkey.Close() + break + + def getmod(self, nm): + stuff = self.map.get(nm) + if stuff: + fnm, desc = stuff + fp = open(fnm, 'rb') + mod = imp.load_module(nm, fp, fnm, desc) + mod.__file__ = fnm + return mod + return None + +class PathImportDirector(ImportDirector): + """Directs imports of modules stored on the filesystem.""" + + def __init__(self, pathlist=None, importers=None, ownertypes=None): + if pathlist is None: + self.path = sys.path + else: + self.path = pathlist + if ownertypes == None: + self._ownertypes = _globalOwnerTypes + else: + self._ownertypes = ownertypes + if importers: + self._shadowPath = importers + else: + self._shadowPath = {} + self._inMakeOwner = False + self._building = {} + + def getmod(self, nm): + mod = None + for thing in self.path: + if isinstance(thing, basestring): + owner = self._shadowPath.get(thing, -1) + if owner == -1: + owner = self._shadowPath[thing] = self._makeOwner(thing) + if owner: + mod = owner.getmod(nm) + else: + mod = thing.getmod(nm) + if mod: + break + return mod + + def _makeOwner(self, path): + if self._building.get(path): + return None + self._building[path] = 1 + owner = None + for klass in self._ownertypes: + try: + # this may cause an import, which may cause recursion + # hence the protection + owner = klass(path) + except: + pass + else: + break + del self._building[path] + return owner + +#=================ImportManager============================# +# The one-and-only ImportManager +# ie, the builtin import + +UNTRIED = -1 + +class ImportManager: + # really the equivalent of builtin import + def __init__(self): + self.metapath = [ + BuiltinImportDirector(), + FrozenImportDirector(), + RegistryImportDirector(), + PathImportDirector() + ] + self.threaded = 0 + self.rlock = None + self.locker = None + self.setThreaded() + + def setThreaded(self): + thread = sys.modules.get('thread', None) + if thread and not self.threaded: + self.threaded = 1 + self.rlock = thread.allocate_lock() + self._get_ident = thread.get_ident + + def install(self): + import __builtin__ + __builtin__.__import__ = self.importHook + __builtin__.reload = self.reloadHook + + def importHook(self, name, globals=None, locals=None, fromlist=None, level=-1): + ''' + NOTE: Currently importHook will accept the keyword-argument "level" + but it will *NOT* use it (currently). Details about the "level" keyword + argument can be found here: http://www.python.org/doc/2.5.2/lib/built-in-funcs.html + ''' + # first see if we could be importing a relative name + #print "importHook(%s, %s, locals, %s)" % (name, globals['__name__'], fromlist) + _sys_modules_get = sys.modules.get + contexts = [None] + if globals: + importernm = globals.get('__name__', '') + if importernm: + if hasattr(_sys_modules_get(importernm), '__path__'): + contexts.insert(0, importernm) + else: + pkgnm = packageName(importernm) + if pkgnm: + contexts.insert(0, pkgnm) + # so contexts is [pkgnm, None] or just [None] + # now break the name being imported up so we get: + # a.b.c -> [a, b, c] + nmparts = nameSplit(name) + _self_doimport = self.doimport + threaded = self.threaded + for context in contexts: + ctx = context + for i in range(len(nmparts)): + nm = nmparts[i] + #print " importHook trying %s in %s" % (nm, ctx) + if ctx: + fqname = ctx + '.' + nm + else: + fqname = nm + if threaded: + self._acquire() + mod = _sys_modules_get(fqname, UNTRIED) + if mod is UNTRIED: + mod = _self_doimport(nm, ctx, fqname) + if threaded: + self._release() + if mod: + ctx = fqname + else: + break + else: + # no break, point i beyond end + i = i + 1 + if i: + break + + if i= len(fromlist): + break + nm = fromlist[i] + i = i + 1 + if not hasattr(bottommod, nm): + if self.threaded: + self._acquire() + mod = self.doimport(nm, ctx, ctx+'.'+nm) + if self.threaded: + self._release() + if not mod: + raise ImportError("%s not found in %s" % (nm, ctx)) + #print "importHook done with %s %s %s (case 3)" % (name, globals['__name__'], fromlist) + return bottommod + + def doimport(self, nm, parentnm, fqname): + # Not that nm is NEVER a dotted name at this point + #print "doimport(%s, %s, %s)" % (nm, parentnm, fqname) + if parentnm: + parent = sys.modules[parentnm] + if hasattr(parent, '__path__'): + importfunc = getattr(parent, '__importsub__', None) + if not importfunc: + subimporter = PathImportDirector(parent.__path__) + importfunc = parent.__importsub__ = subimporter.getmod + mod = importfunc(nm) + if mod: + setattr(parent, nm, mod) + else: + #print "..parent not a package" + return None + else: + # now we're dealing with an absolute import + for director in self.metapath: + mod = director.getmod(nm) + if mod: + break + if mod: + mod.__name__ = fqname + sys.modules[fqname] = mod + if hasattr(mod, '__co__'): + co = mod.__co__ + del mod.__co__ + exec(co, mod.__dict__) + if fqname == 'thread' and not self.threaded: +## print "thread detected!" + self.setThreaded() + else: + sys.modules[fqname] = None + #print "..found %s" % mod + return mod + + def reloadHook(self, mod): + fqnm = mod.__name__ + nm = nameSplit(fqnm)[-1] + parentnm = packageName(fqnm) + newmod = self.doimport(nm, parentnm, fqnm) + mod.__dict__.update(newmod.__dict__) +## return newmod + + def _acquire(self): + if self.rlock.locked(): + if self.locker == self._get_ident(): + self.lockcount = self.lockcount + 1 +## print "_acquire incrementing lockcount to", self.lockcount + return + self.rlock.acquire() + self.locker = self._get_ident() + self.lockcount = 0 +## print "_acquire first time!" + + def _release(self): + if self.lockcount: + self.lockcount = self.lockcount - 1 +## print "_release decrementing lockcount to", self.lockcount + else: + self.rlock.release() +## print "_release releasing lock!" + + +################################################## +## MORE CONSTANTS & GLOBALS + +_globalOwnerTypes = [ + DirOwner, + Owner, +] === added directory 'cheetah/Macros' === added file 'cheetah/Macros/I18n.py' --- cheetah/Macros/I18n.py 1970-01-01 00:00:00 +0000 +++ cheetah/Macros/I18n.py 2010-10-19 19:11:10 +0000 @@ -0,0 +1,67 @@ +import gettext +_ = gettext.gettext +class I18n(object): + def __init__(self, parser): + pass + +## junk I'm playing with to test the macro framework +# def parseArgs(self, parser, startPos): +# parser.getWhiteSpace() +# args = parser.getExpression(useNameMapper=False, +# pyTokensToBreakAt=[':']).strip() +# return args +# +# def convertArgStrToDict(self, args, parser=None, startPos=None): +# def getArgs(*pargs, **kws): +# return pargs, kws +# exec 'positionalArgs, kwArgs = getArgs(%(args)s)'%locals() +# return kwArgs + + def __call__(self, + src, # aka message, + plural=None, + n=None, # should be a string representing the name of the + # '$var' rather than $var itself + id=None, + domain=None, + source=None, + target=None, + comment=None, + + # args that are automatically supplied by the parser when the + # macro is called: + parser=None, + macros=None, + isShortForm=False, + EOLCharsInShortForm=None, + startPos=None, + endPos=None, + ): + """This is just a stub at this time. + + plural = the plural form of the message + n = a sized argument to distinguish between single and plural forms + + id = msgid in the translation catalog + domain = translation domain + source = source lang + target = a specific target lang + comment = a comment to the translation team + + See the following for some ideas + http://www.zope.org/DevHome/Wikis/DevSite/Projects/ComponentArchitecture/ZPTInternationalizationSupport + + Other notes: + - There is no need to replicate the i18n:name attribute from plone / PTL, + as cheetah placeholders serve the same purpose + + + """ + + #print macros['i18n'] + src = _(src) + if isShortForm and endPos (42 chars) + +Using Cheetah's NameMapper syntax it could be any of the following: + + $self.customers()[$ID].address()['city'] (39 chars) + --OR-- + $customers()[$ID].address()['city'] + --OR-- + $customers()[$ID].address().city + --OR-- + $customers()[$ID].address.city + --OR-- + $customers()[$ID].address.city + --OR-- + $customers[$ID].address.city (27 chars) + + +Which of these would you prefer to explain to the designers, who have no +programming experience? The last form is 15 characters shorter than the PSP +and, conceptually, is far more accessible. With PHP or ASP, the code would be +even messier than the PSP + +This is a rather extreme example and, of course, you could also just implement +'$getCustomer($ID).city' and obey the Law of Demeter (search Google for more on that). +But good object orientated design isn't the point here. + +Details +================================================================================ +The parenthesized letters below correspond to the aims in the second paragraph. + +DICTIONARY ACCESS (a) +--------------------- + +NameMapper allows access to items in a dictionary using the same dotted notation +used to access object attributes in Python. This aspect of NameMapper is known +as 'Unified Dotted Notation'. + +For example, with Cheetah it is possible to write: + $customers()['kerr'].address() --OR-- $customers().kerr.address() +where the second form is in NameMapper syntax. + +This only works with dictionary keys that are also valid python identifiers: + regex = '[a-zA-Z_][a-zA-Z_0-9]*' + + +AUTOCALLING (b,d) +----------------- + +NameMapper automatically detects functions and methods in Cheetah $vars and calls +them if the parentheses have been left off. + +For example if 'a' is an object, 'b' is a method + $a.b +is equivalent to + $a.b() + +If b returns a dictionary, then following variations are possible + $a.b.c --OR-- $a.b().c --OR-- $a.b()['c'] +where 'c' is a key in the dictionary that a.b() returns. + +Further notes: +* NameMapper autocalls the function or method without any arguments. Thus +autocalling can only be used with functions or methods that either have no +arguments or have default values for all arguments. + +* NameMapper only autocalls functions and methods. Classes and callable object instances +will not be autocalled. + +* Autocalling can be disabled using Cheetah's 'useAutocalling' setting. + +LEAVING OUT 'self' (c,d) +------------------------ + +NameMapper makes it possible to access the attributes of a servlet in Cheetah +without needing to include 'self' in the variable names. See the NAMESPACE +CASCADING section below for details. + +NAMESPACE CASCADING (d) +-------------------- +... + +Implementation details +================================================================================ + +* NameMapper's search order is dictionary keys then object attributes + +* NameMapper.NotFound is raised if a value can't be found for a name. + +Performance and the C version +================================================================================ + +Cheetah comes with both a C version and a Python version of NameMapper. The C +version is significantly faster and the exception tracebacks are much easier to +read. It's still slower than standard Python syntax, but you won't notice the +difference in realistic usage scenarios. + +Cheetah uses the optimized C version (_namemapper.c) if it has +been compiled or falls back to the Python version if not. +""" + +__author__ = "Tavis Rudd ," +\ + "\nChuck Esterbrook " +from pprint import pformat +import inspect + +_INCLUDE_NAMESPACE_REPR_IN_NOTFOUND_EXCEPTIONS = False +_ALLOW_WRAPPING_OF_NOTFOUND_EXCEPTIONS = True +__all__ = ['NotFound', + 'hasKey', + 'valueForKey', + 'valueForName', + 'valueFromSearchList', + 'valueFromFrameOrSearchList', + 'valueFromFrame', + ] + +if not hasattr(inspect.imp, 'get_suffixes'): + # This is to fix broken behavior of the inspect module under the + # Google App Engine, see the following issue: + # http://bugs.communitycheetah.org/view.php?id=10 + setattr(inspect.imp, 'get_suffixes', lambda: [('.py', 'U', 1)]) + +## N.B. An attempt is made at the end of this module to import C versions of +## these functions. If _namemapper.c has been compiled succesfully and the +## import goes smoothly, the Python versions defined here will be replaced with +## the C versions. + +class NotFound(LookupError): + pass + +def _raiseNotFoundException(key, namespace): + excString = "cannot find '%s'"%key + if _INCLUDE_NAMESPACE_REPR_IN_NOTFOUND_EXCEPTIONS: + excString += ' in the namespace %s'%pformat(namespace) + raise NotFound(excString) + +def _wrapNotFoundException(exc, fullName, namespace): + if not _ALLOW_WRAPPING_OF_NOTFOUND_EXCEPTIONS: + raise + else: + excStr = exc.args[0] + if excStr.find('while searching')==-1: # only wrap once! + excStr +=" while searching for '%s'"%fullName + if _INCLUDE_NAMESPACE_REPR_IN_NOTFOUND_EXCEPTIONS: + excStr += ' in the namespace %s'%pformat(namespace) + exc.args = (excStr,) + raise + +def _isInstanceOrClass(obj): + if isinstance(obj, type): + # oldstyle + return True + + if hasattr(obj, "__class__"): + # newstyle + if hasattr(obj, 'mro'): + # type/class + return True + elif (hasattr(obj, 'im_func') or hasattr(obj, 'func_code') or hasattr(obj, '__self__')): + # method, func, or builtin func + return False + elif hasattr(obj, '__init__'): + # instance + return True + return False + +def hasKey(obj, key): + """Determine if 'obj' has 'key' """ + if hasattr(obj, 'has_key') and key in obj: + return True + elif hasattr(obj, key): + return True + else: + return False + +def valueForKey(obj, key): + if hasattr(obj, 'has_key') and key in obj: + return obj[key] + elif hasattr(obj, key): + return getattr(obj, key) + else: + _raiseNotFoundException(key, obj) + +def _valueForName(obj, name, executeCallables=False): + nameChunks=name.split('.') + for i in range(len(nameChunks)): + key = nameChunks[i] + if hasattr(obj, 'has_key') and key in obj: + nextObj = obj[key] + else: + try: + nextObj = getattr(obj, key) + except AttributeError: + _raiseNotFoundException(key, obj) + + if executeCallables and hasattr(nextObj, '__call__') and not _isInstanceOrClass(nextObj): + obj = nextObj() + else: + obj = nextObj + return obj + +def valueForName(obj, name, executeCallables=False): + try: + return _valueForName(obj, name, executeCallables) + except NotFound, e: + _wrapNotFoundException(e, fullName=name, namespace=obj) + +def valueFromSearchList(searchList, name, executeCallables=False): + key = name.split('.')[0] + for namespace in searchList: + if hasKey(namespace, key): + return _valueForName(namespace, name, + executeCallables=executeCallables) + _raiseNotFoundException(key, searchList) + +def _namespaces(callerFrame, searchList=None): + yield callerFrame.f_locals + if searchList: + for namespace in searchList: + yield namespace + yield callerFrame.f_globals + yield __builtins__ + +def valueFromFrameOrSearchList(searchList, name, executeCallables=False, + frame=None): + def __valueForName(): + try: + return _valueForName(namespace, name, executeCallables=executeCallables) + except NotFound, e: + _wrapNotFoundException(e, fullName=name, namespace=searchList) + try: + if not frame: + frame = inspect.stack()[1][0] + key = name.split('.')[0] + for namespace in _namespaces(frame, searchList): + if hasKey(namespace, key): + return __valueForName() + _raiseNotFoundException(key, searchList) + finally: + del frame + +def valueFromFrame(name, executeCallables=False, frame=None): + # @@TR consider implementing the C version the same way + # at the moment it provides a seperate but mirror implementation + # to valueFromFrameOrSearchList + try: + if not frame: + frame = inspect.stack()[1][0] + return valueFromFrameOrSearchList(searchList=None, + name=name, + executeCallables=executeCallables, + frame=frame) + finally: + del frame + +def hasName(obj, name): + #Not in the C version + """Determine if 'obj' has the 'name' """ + key = name.split('.')[0] + if not hasKey(obj, key): + return False + try: + valueForName(obj, name) + return True + except NotFound: + return False +try: + from Cheetah._namemapper import NotFound, valueForKey, valueForName, \ + valueFromSearchList, valueFromFrameOrSearchList, valueFromFrame + # it is possible with Jython or Windows, for example, that _namemapper.c hasn't been compiled + C_VERSION = True +except: + C_VERSION = False + +################################################## +## CLASSES + +class Mixin: + """@@ document me""" + def valueForName(self, name): + return valueForName(self, name) + + def valueForKey(self, key): + return valueForKey(self, key) + +################################################## +## if run from the command line ## + +def example(): + class A(Mixin): + classVar = 'classVar val' + def method(self,arg='method 1 default arg'): + return arg + + def method2(self, arg='meth 2 default arg'): + return {'item1':arg} + + def method3(self, arg='meth 3 default'): + return arg + + class B(A): + classBvar = 'classBvar val' + + a = A() + a.one = 'valueForOne' + def function(whichOne='default'): + values = { + 'default': 'default output', + 'one': 'output option one', + 'two': 'output option two' + } + return values[whichOne] + + a.dic = { + 'func': function, + 'method': a.method3, + 'item': 'itemval', + 'subDict': {'nestedMethod':a.method3} + } + b = 'this is local b' + + print(valueForKey(a.dic, 'subDict')) + print(valueForName(a, 'dic.item')) + print(valueForName(vars(), 'b')) + print(valueForName(__builtins__, 'dir')()) + print(valueForName(vars(), 'a.classVar')) + print(valueForName(vars(), 'a.dic.func', executeCallables=True)) + print(valueForName(vars(), 'a.method2.item1', executeCallables=True)) + +if __name__ == '__main__': + example() + + + === added file 'cheetah/Parser.py' --- cheetah/Parser.py 1970-01-01 00:00:00 +0000 +++ cheetah/Parser.py 2010-10-19 19:11:10 +0000 @@ -0,0 +1,2661 @@ +#!/usr/bin/env python +""" +Parser classes for Cheetah's Compiler + +Classes: + ParseError( Exception ) + _LowLevelParser( Cheetah.SourceReader.SourceReader ), basically a lexer + _HighLevelParser( _LowLevelParser ) + Parser === _HighLevelParser (an alias) +""" + +import os +import sys +import re +from re import DOTALL, MULTILINE +import types +import time +from tokenize import pseudoprog +import inspect +import traceback + +from Cheetah.SourceReader import SourceReader +from Cheetah import Filters +from Cheetah import ErrorCatchers +from Cheetah.Unspecified import Unspecified +from Cheetah.Macros.I18n import I18n + +# re tools +_regexCache = {} +def cachedRegex(pattern): + if pattern not in _regexCache: + _regexCache[pattern] = re.compile(pattern) + return _regexCache[pattern] + +def escapeRegexChars(txt, + escapeRE=re.compile(r'([\$\^\*\+\.\?\{\}\[\]\(\)\|\\])')): + + """Return a txt with all special regular expressions chars escaped.""" + + return escapeRE.sub(r'\\\1', txt) + +def group(*choices): return '(' + '|'.join(choices) + ')' +def nongroup(*choices): return '(?:' + '|'.join(choices) + ')' +def namedGroup(name, *choices): return '(P:<' + name +'>' + '|'.join(choices) + ')' +def any(*choices): return group(*choices) + '*' +def maybe(*choices): return group(*choices) + '?' + +################################################## +## CONSTANTS & GLOBALS ## + +NO_CACHE = 0 +STATIC_CACHE = 1 +REFRESH_CACHE = 2 + +SET_LOCAL = 0 +SET_GLOBAL = 1 +SET_MODULE = 2 + +################################################## +## Tokens for the parser ## + +#generic +identchars = "abcdefghijklmnopqrstuvwxyz" \ + "ABCDEFGHIJKLMNOPQRSTUVWXYZ_" +namechars = identchars + "0123456789" + +#operators +powerOp = '**' +unaryArithOps = ('+', '-', '~') +binaryArithOps = ('+', '-', '/', '//', '%') +shiftOps = ('>>', '<<') +bitwiseOps = ('&', '|', '^') +assignOp = '=' +augAssignOps = ('+=', '-=', '/=', '*=', '**=', '^=', '%=', + '>>=', '<<=', '&=', '|=', ) +assignmentOps = (assignOp,) + augAssignOps + +compOps = ('<', '>', '==', '!=', '<=', '>=', '<>', 'is', 'in',) +booleanOps = ('and', 'or', 'not') +operators = (powerOp,) + unaryArithOps + binaryArithOps \ + + shiftOps + bitwiseOps + assignmentOps \ + + compOps + booleanOps + +delimeters = ('(', ')', '{', '}', '[', ']', + ',', '.', ':', ';', '=', '`') + augAssignOps + + +keywords = ('and', 'del', 'for', 'is', 'raise', + 'assert', 'elif', 'from', 'lambda', 'return', + 'break', 'else', 'global', 'not', 'try', + 'class', 'except', 'if', 'or', 'while', + 'continue', 'exec', 'import', 'pass', + 'def', 'finally', 'in', 'print', + ) + +single3 = "'''" +double3 = '"""' + +tripleQuotedStringStarts = ("'''", '"""', + "r'''", 'r"""', "R'''", 'R"""', + "u'''", 'u"""', "U'''", 'U"""', + "ur'''", 'ur"""', "Ur'''", 'Ur"""', + "uR'''", 'uR"""', "UR'''", 'UR"""') + +tripleQuotedStringPairs = {"'''": single3, '"""': double3, + "r'''": single3, 'r"""': double3, + "u'''": single3, 'u"""': double3, + "ur'''": single3, 'ur"""': double3, + "R'''": single3, 'R"""': double3, + "U'''": single3, 'U"""': double3, + "uR'''": single3, 'uR"""': double3, + "Ur'''": single3, 'Ur"""': double3, + "UR'''": single3, 'UR"""': double3, + } + +closurePairs= {')':'(',']':'[','}':'{'} +closurePairsRev= {'(':')','[':']','{':'}'} + +################################################## +## Regex chunks for the parser ## + +tripleQuotedStringREs = {} +def makeTripleQuoteRe(start, end): + start = escapeRegexChars(start) + end = escapeRegexChars(end) + return re.compile(r'(?:' + start + r').*?' + r'(?:' + end + r')', re.DOTALL) + +for start, end in tripleQuotedStringPairs.items(): + tripleQuotedStringREs[start] = makeTripleQuoteRe(start, end) + +WS = r'[ \f\t]*' +EOL = r'\r\n|\n|\r' +EOLZ = EOL + r'|\Z' +escCharLookBehind = nongroup(r'(?<=\A)', r'(?= len(stream): + stream.setPos(len(stream) -1) + self.msg = msg + self.extMsg = extMsg + self.lineno = lineno + self.col = col + + def __str__(self): + return self.report() + + def report(self): + stream = self.stream + if stream.filename(): + f = " in file %s" % stream.filename() + else: + f = '' + report = '' + if self.lineno: + lineno = self.lineno + row, col, line = (lineno, (self.col or 0), + self.stream.splitlines()[lineno-1]) + else: + row, col, line = self.stream.getRowColLine() + + ## get the surrounding lines + lines = stream.splitlines() + prevLines = [] # (rowNum, content) + for i in range(1, 4): + if row-1-i <=0: + break + prevLines.append( (row-i, lines[row-1-i]) ) + + nextLines = [] # (rowNum, content) + for i in range(1, 4): + if not row-1+i < len(lines): + break + nextLines.append( (row+i, lines[row-1+i]) ) + nextLines.reverse() + + ## print the main message + report += "\n\n%s\n" %self.msg + report += "Line %i, column %i%s\n\n" % (row, col, f) + report += 'Line|Cheetah Code\n' + report += '----|-------------------------------------------------------------\n' + while prevLines: + lineInfo = prevLines.pop() + report += "%(row)-4d|%(line)s\n"% {'row':lineInfo[0], 'line':lineInfo[1]} + report += "%(row)-4d|%(line)s\n"% {'row':row, 'line':line} + report += ' '*5 +' '*(col-1) + "^\n" + + while nextLines: + lineInfo = nextLines.pop() + report += "%(row)-4d|%(line)s\n"% {'row':lineInfo[0], 'line':lineInfo[1]} + ## add the extra msg + if self.extMsg: + report += self.extMsg + '\n' + + return report + +class ForbiddenSyntax(ParseError): + pass +class ForbiddenExpression(ForbiddenSyntax): + pass +class ForbiddenDirective(ForbiddenSyntax): + pass + +class CheetahVariable(object): + def __init__(self, nameChunks, useNameMapper=True, cacheToken=None, + rawSource=None): + self.nameChunks = nameChunks + self.useNameMapper = useNameMapper + self.cacheToken = cacheToken + self.rawSource = rawSource + +class Placeholder(CheetahVariable): + pass + +class ArgList(object): + """Used by _LowLevelParser.getArgList()""" + + def __init__(self): + self.arguments = [] + self.defaults = [] + self.count = 0 + + def add_argument(self, name): + self.arguments.append(name) + self.defaults.append(None) + + def next(self): + self.count += 1 + + def add_default(self, token): + count = self.count + if self.defaults[count] is None: + self.defaults[count] = '' + self.defaults[count] += token + + def merge(self): + defaults = (isinstance(d, basestring) and d.strip() or None for d in self.defaults) + return list(map(None, (a.strip() for a in self.arguments), defaults)) + + def __str__(self): + return str(self.merge()) + +class _LowLevelParser(SourceReader): + """This class implements the methods to match or extract ('get*') the basic + elements of Cheetah's grammar. It does NOT handle any code generation or + state management. + """ + + _settingsManager = None + + def setSettingsManager(self, settingsManager): + self._settingsManager = settingsManager + + def setting(self, key, default=Unspecified): + if default is Unspecified: + return self._settingsManager.setting(key) + else: + return self._settingsManager.setting(key, default=default) + + def setSetting(self, key, val): + self._settingsManager.setSetting(key, val) + + def settings(self): + return self._settingsManager.settings() + + def updateSettings(self, settings): + self._settingsManager.updateSettings(settings) + + def _initializeSettings(self): + self._settingsManager._initializeSettings() + + def configureParser(self): + """Is called by the Compiler instance after the parser has had a + settingsManager assigned with self.setSettingsManager() + """ + self._makeCheetahVarREs() + self._makeCommentREs() + self._makeDirectiveREs() + self._makePspREs() + self._possibleNonStrConstantChars = ( + self.setting('commentStartToken')[0] + + self.setting('multiLineCommentStartToken')[0] + + self.setting('cheetahVarStartToken')[0] + + self.setting('directiveStartToken')[0] + + self.setting('PSPStartToken')[0]) + self._nonStrConstMatchers = [ + self.matchCommentStartToken, + self.matchMultiLineCommentStartToken, + self.matchVariablePlaceholderStart, + self.matchExpressionPlaceholderStart, + self.matchDirective, + self.matchPSPStartToken, + self.matchEOLSlurpToken, + ] + + ## regex setup ## + + def _makeCheetahVarREs(self): + + """Setup the regexs for Cheetah $var parsing.""" + + num = r'[0-9\.]+' + interval = (r'(?P' + + num + r's|' + + num + r'm|' + + num + r'h|' + + num + r'd|' + + num + r'w|' + + num + ')' + ) + + cacheToken = (r'(?:' + + r'(?P\*' + interval + '\*)'+ + '|' + + r'(?P\*)' + + '|' + + r'(?P)' + + ')') + self.cacheTokenRE = cachedRegex(cacheToken) + + silentPlaceholderToken = (r'(?:' + + r'(?P' +escapeRegexChars('!')+')'+ + '|' + + r'(?P)' + + ')') + self.silentPlaceholderTokenRE = cachedRegex(silentPlaceholderToken) + + self.cheetahVarStartRE = cachedRegex( + escCharLookBehind + + r'(?P'+escapeRegexChars(self.setting('cheetahVarStartToken'))+')'+ + r'(?P'+silentPlaceholderToken+')'+ + r'(?P'+cacheToken+')'+ + r'(?P|(?:(?:\{|\(|\[)[ \t\f]*))' + # allow WS after enclosure + r'(?=[A-Za-z_])') + validCharsLookAhead = r'(?=[A-Za-z_\*!\{\(\[])' + self.cheetahVarStartToken = self.setting('cheetahVarStartToken') + self.cheetahVarStartTokenRE = cachedRegex( + escCharLookBehind + + escapeRegexChars(self.setting('cheetahVarStartToken')) + +validCharsLookAhead + ) + + self.cheetahVarInExpressionStartTokenRE = cachedRegex( + escapeRegexChars(self.setting('cheetahVarStartToken')) + +r'(?=[A-Za-z_])' + ) + + self.expressionPlaceholderStartRE = cachedRegex( + escCharLookBehind + + r'(?P' + escapeRegexChars(self.setting('cheetahVarStartToken')) + ')' + + r'(?P' + cacheToken + ')' + + #r'\[[ \t\f]*' + r'(?:\{|\(|\[)[ \t\f]*' + + r'(?=[^\)\}\]])' + ) + + if self.setting('EOLSlurpToken'): + self.EOLSlurpRE = cachedRegex( + escapeRegexChars(self.setting('EOLSlurpToken')) + + r'[ \t\f]*' + + r'(?:'+EOL+')' + ) + else: + self.EOLSlurpRE = None + + + def _makeCommentREs(self): + """Construct the regex bits that are used in comment parsing.""" + startTokenEsc = escapeRegexChars(self.setting('commentStartToken')) + self.commentStartTokenRE = cachedRegex(escCharLookBehind + startTokenEsc) + del startTokenEsc + + startTokenEsc = escapeRegexChars( + self.setting('multiLineCommentStartToken')) + endTokenEsc = escapeRegexChars( + self.setting('multiLineCommentEndToken')) + self.multiLineCommentTokenStartRE = cachedRegex(escCharLookBehind + + startTokenEsc) + self.multiLineCommentEndTokenRE = cachedRegex(escCharLookBehind + + endTokenEsc) + + def _makeDirectiveREs(self): + """Construct the regexs that are used in directive parsing.""" + startToken = self.setting('directiveStartToken') + endToken = self.setting('directiveEndToken') + startTokenEsc = escapeRegexChars(startToken) + endTokenEsc = escapeRegexChars(endToken) + validSecondCharsLookAhead = r'(?=[A-Za-z_@])' + reParts = [escCharLookBehind, startTokenEsc] + if self.setting('allowWhitespaceAfterDirectiveStartToken'): + reParts.append('[ \t]*') + reParts.append(validSecondCharsLookAhead) + self.directiveStartTokenRE = cachedRegex(''.join(reParts)) + self.directiveEndTokenRE = cachedRegex(escCharLookBehind + endTokenEsc) + + def _makePspREs(self): + """Setup the regexs for PSP parsing.""" + startToken = self.setting('PSPStartToken') + startTokenEsc = escapeRegexChars(startToken) + self.PSPStartTokenRE = cachedRegex(escCharLookBehind + startTokenEsc) + endToken = self.setting('PSPEndToken') + endTokenEsc = escapeRegexChars(endToken) + self.PSPEndTokenRE = cachedRegex(escCharLookBehind + endTokenEsc) + + def _unescapeCheetahVars(self, theString): + """Unescape any escaped Cheetah \$vars in the string. + """ + + token = self.setting('cheetahVarStartToken') + return theString.replace('\\' + token, token) + + def _unescapeDirectives(self, theString): + """Unescape any escaped Cheetah directives in the string. + """ + + token = self.setting('directiveStartToken') + return theString.replace('\\' + token, token) + + def isLineClearToStartToken(self, pos=None): + return self.isLineClearToPos(pos) + + def matchTopLevelToken(self): + """Returns the first match found from the following methods: + self.matchCommentStartToken + self.matchMultiLineCommentStartToken + self.matchVariablePlaceholderStart + self.matchExpressionPlaceholderStart + self.matchDirective + self.matchPSPStartToken + self.matchEOLSlurpToken + + Returns None if no match. + """ + match = None + if self.peek() in self._possibleNonStrConstantChars: + for matcher in self._nonStrConstMatchers: + match = matcher() + if match: + break + return match + + def matchPyToken(self): + match = pseudoprog.match(self.src(), self.pos()) + + if match and match.group() in tripleQuotedStringStarts: + TQSmatch = tripleQuotedStringREs[match.group()].match(self.src(), self.pos()) + if TQSmatch: + return TQSmatch + return match + + def getPyToken(self): + match = self.matchPyToken() + if match is None: + raise ParseError(self) + elif match.group() in tripleQuotedStringStarts: + raise ParseError(self, msg='Malformed triple-quoted string') + return self.readTo(match.end()) + + def matchEOLSlurpToken(self): + if self.EOLSlurpRE: + return self.EOLSlurpRE.match(self.src(), self.pos()) + + def getEOLSlurpToken(self): + match = self.matchEOLSlurpToken() + if not match: + raise ParseError(self, msg='Invalid EOL slurp token') + return self.readTo(match.end()) + + def matchCommentStartToken(self): + return self.commentStartTokenRE.match(self.src(), self.pos()) + + def getCommentStartToken(self): + match = self.matchCommentStartToken() + if not match: + raise ParseError(self, msg='Invalid single-line comment start token') + return self.readTo(match.end()) + + def matchMultiLineCommentStartToken(self): + return self.multiLineCommentTokenStartRE.match(self.src(), self.pos()) + + def getMultiLineCommentStartToken(self): + match = self.matchMultiLineCommentStartToken() + if not match: + raise ParseError(self, msg='Invalid multi-line comment start token') + return self.readTo(match.end()) + + def matchMultiLineCommentEndToken(self): + return self.multiLineCommentEndTokenRE.match(self.src(), self.pos()) + + def getMultiLineCommentEndToken(self): + match = self.matchMultiLineCommentEndToken() + if not match: + raise ParseError(self, msg='Invalid multi-line comment end token') + return self.readTo(match.end()) + + def getCommaSeparatedSymbols(self): + """ + Loosely based on getDottedName to pull out comma separated + named chunks + """ + srcLen = len(self) + pieces = [] + nameChunks = [] + + if not self.peek() in identchars: + raise ParseError(self) + + while self.pos() < srcLen: + c = self.peek() + if c in namechars: + nameChunk = self.getIdentifier() + nameChunks.append(nameChunk) + elif c == '.': + if self.pos()+1 endOfFirstLine): + self._compiler.handleWSBeforeDirective() + + self._compiler.addComment(comm) + + def eatPlaceholder(self): + (expr, rawPlaceholder, + lineCol, cacheTokenParts, + filterArgs, isSilentPlaceholder) = self.getPlaceholder( + allowCacheTokens=True, returnEverything=True) + + self._compiler.addPlaceholder( + expr, + filterArgs=filterArgs, + rawPlaceholder=rawPlaceholder, + cacheTokenParts=cacheTokenParts, + lineCol=lineCol, + silentMode=isSilentPlaceholder) + return + + def eatPSP(self): + # filtered + self._filterDisabledDirectives(directiveName='psp') + self.getPSPStartToken() + endToken = self.setting('PSPEndToken') + startPos = self.pos() + while not self.atEnd(): + if self.peek() == endToken[0]: + if self.matchPSPEndToken(): + break + self.advance() + pspString = self.readTo(self.pos(), start=startPos).strip() + pspString = self._applyExpressionFilters(pspString, 'psp', startPos=startPos) + self._compiler.addPSP(pspString) + self.getPSPEndToken() + + ## generic directive eat methods + _simpleIndentingDirectives = ''' + else elif for while repeat unless try except finally'''.split() + _simpleExprDirectives = ''' + pass continue stop return yield break + del assert raise + silent echo + import from'''.split() + _directiveHandlerNames = {'import': 'addImportStatement', + 'from': 'addImportStatement', } + def eatDirective(self): + directiveName = self.matchDirective() + self._filterDisabledDirectives(directiveName) + + for callback in self.setting('preparseDirectiveHooks'): + callback(parser=self, directiveName=directiveName) + + # subclasses can override the default behaviours here by providing an + # eater method in self._directiveNamesAndParsers[directiveName] + directiveParser = self._directiveNamesAndParsers.get(directiveName) + if directiveParser: + directiveParser() + elif directiveName in self._simpleIndentingDirectives: + handlerName = self._directiveHandlerNames.get(directiveName) + if not handlerName: + handlerName = 'add'+directiveName.capitalize() + handler = getattr(self._compiler, handlerName) + self.eatSimpleIndentingDirective(directiveName, callback=handler) + elif directiveName in self._simpleExprDirectives: + handlerName = self._directiveHandlerNames.get(directiveName) + if not handlerName: + handlerName = 'add'+directiveName.capitalize() + handler = getattr(self._compiler, handlerName) + if directiveName in ('silent', 'echo'): + includeDirectiveNameInExpr = False + else: + includeDirectiveNameInExpr = True + expr = self.eatSimpleExprDirective( + directiveName, + includeDirectiveNameInExpr=includeDirectiveNameInExpr) + handler(expr) + ## + for callback in self.setting('postparseDirectiveHooks'): + callback(parser=self, directiveName=directiveName) + + def _eatRestOfDirectiveTag(self, isLineClearToStartToken, endOfFirstLinePos): + foundComment = False + if self.matchCommentStartToken(): + pos = self.pos() + self.advance() + if not self.matchDirective(): + self.setPos(pos) + foundComment = True + self.eatComment() # this won't gobble the EOL + else: + self.setPos(pos) + + if not foundComment and self.matchDirectiveEndToken(): + self.getDirectiveEndToken() + elif isLineClearToStartToken and (not self.atEnd()) and self.peek() in '\r\n': + # still gobble the EOL if a comment was found. + self.readToEOL(gobble=True) + + if isLineClearToStartToken and (self.atEnd() or self.pos() > endOfFirstLinePos): + self._compiler.handleWSBeforeDirective() + + def _eatToThisEndDirective(self, directiveName): + finalPos = endRawPos = startPos = self.pos() + directiveChar = self.setting('directiveStartToken')[0] + isLineClearToStartToken = False + while not self.atEnd(): + if self.peek() == directiveChar: + if self.matchDirective() == 'end': + endRawPos = self.pos() + self.getDirectiveStartToken() + self.advance(len('end')) + self.getWhiteSpace() + if self.startswith(directiveName): + if self.isLineClearToStartToken(endRawPos): + isLineClearToStartToken = True + endRawPos = self.findBOL(endRawPos) + self.advance(len(directiveName)) # to end of directiveName + self.getWhiteSpace() + finalPos = self.pos() + break + self.advance() + finalPos = endRawPos = self.pos() + + textEaten = self.readTo(endRawPos, start=startPos) + self.setPos(finalPos) + + endOfFirstLinePos = self.findEOL() + + if self.matchDirectiveEndToken(): + self.getDirectiveEndToken() + elif isLineClearToStartToken and (not self.atEnd()) and self.peek() in '\r\n': + self.readToEOL(gobble=True) + + if isLineClearToStartToken and self.pos() > endOfFirstLinePos: + self._compiler.handleWSBeforeDirective() + return textEaten + + + def eatSimpleExprDirective(self, directiveName, includeDirectiveNameInExpr=True): + # filtered + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLine = self.findEOL() + self.getDirectiveStartToken() + if not includeDirectiveNameInExpr: + self.advance(len(directiveName)) + startPos = self.pos() + expr = self.getExpression().strip() + directiveName = expr.split()[0] + expr = self._applyExpressionFilters(expr, directiveName, startPos=startPos) + if directiveName in self._closeableDirectives: + self.pushToOpenDirectivesStack(directiveName) + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLine) + return expr + + def eatSimpleIndentingDirective(self, directiveName, callback, + includeDirectiveNameInExpr=False): + # filtered + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLinePos = self.findEOL() + lineCol = self.getRowCol() + self.getDirectiveStartToken() + if directiveName not in 'else elif for while try except finally'.split(): + self.advance(len(directiveName)) + startPos = self.pos() + + self.getWhiteSpace() + + expr = self.getExpression(pyTokensToBreakAt=[':']) + expr = self._applyExpressionFilters(expr, directiveName, startPos=startPos) + if self.matchColonForSingleLineShortFormDirective(): + self.advance() # skip over : + if directiveName in 'else elif except finally'.split(): + callback(expr, dedent=False, lineCol=lineCol) + else: + callback(expr, lineCol=lineCol) + + self.getWhiteSpace(max=1) + self.parse(breakPoint=self.findEOL(gobble=True)) + self._compiler.commitStrConst() + self._compiler.dedent() + else: + if self.peek()==':': + self.advance() + self.getWhiteSpace() + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLinePos) + if directiveName in self._closeableDirectives: + self.pushToOpenDirectivesStack(directiveName) + callback(expr, lineCol=lineCol) + + def eatEndDirective(self): + isLineClearToStartToken = self.isLineClearToStartToken() + self.getDirectiveStartToken() + self.advance(3) # to end of 'end' + self.getWhiteSpace() + pos = self.pos() + directiveName = False + for key in self._endDirectiveNamesAndHandlers.keys(): + if self.find(key, pos) == pos: + directiveName = key + break + if not directiveName: + raise ParseError(self, msg='Invalid end directive') + + endOfFirstLinePos = self.findEOL() + self.getExpression() # eat in any extra comment-like crap + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLinePos) + if directiveName in self._closeableDirectives: + self.popFromOpenDirectivesStack(directiveName) + + # subclasses can override the default behaviours here by providing an + # end-directive handler in self._endDirectiveNamesAndHandlers[directiveName] + if self._endDirectiveNamesAndHandlers.get(directiveName): + handler = self._endDirectiveNamesAndHandlers[directiveName] + handler() + elif directiveName in 'block capture cache call filter errorCatcher'.split(): + if key == 'block': + self._compiler.closeBlock() + elif key == 'capture': + self._compiler.endCaptureRegion() + elif key == 'cache': + self._compiler.endCacheRegion() + elif key == 'call': + self._compiler.endCallRegion() + elif key == 'filter': + self._compiler.closeFilterBlock() + elif key == 'errorCatcher': + self._compiler.turnErrorCatcherOff() + elif directiveName in 'while for if try repeat unless'.split(): + self._compiler.commitStrConst() + self._compiler.dedent() + elif directiveName=='closure': + self._compiler.commitStrConst() + self._compiler.dedent() + # @@TR: temporary hack of useSearchList + self.setSetting('useSearchList', self._useSearchList_orig) + + ## specific directive eat methods + + def eatBreakPoint(self): + """Tells the parser to stop parsing at this point and completely ignore + everything else. + + This is a debugging tool. + """ + self.setBreakPoint(self.pos()) + + def eatShbang(self): + # filtered + self.getDirectiveStartToken() + self.advance(len('shBang')) + self.getWhiteSpace() + startPos = self.pos() + shBang = self.readToEOL() + shBang = self._applyExpressionFilters(shBang, 'shbang', startPos=startPos) + self._compiler.setShBang(shBang.strip()) + + def eatEncoding(self): + # filtered + self.getDirectiveStartToken() + self.advance(len('encoding')) + self.getWhiteSpace() + startPos = self.pos() + encoding = self.readToEOL() + encoding = self._applyExpressionFilters(encoding, 'encoding', startPos=startPos) + self._compiler.setModuleEncoding(encoding.strip()) + + def eatCompiler(self): + # filtered + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLine = self.findEOL() + startPos = self.pos() + self.getDirectiveStartToken() + self.advance(len('compiler')) # to end of 'compiler' + self.getWhiteSpace() + + startPos = self.pos() + settingName = self.getIdentifier() + + if settingName.lower() == 'reset': + self.getExpression() # gobble whitespace & junk + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLine) + self._initializeSettings() + self.configureParser() + return + + self.getWhiteSpace() + if self.peek() == '=': + self.advance() + else: + raise ParseError(self) + valueExpr = self.getExpression() + endPos = self.pos() + + # @@TR: it's unlikely that anyone apply filters would have left this + # directive enabled: + # @@TR: fix up filtering, regardless + self._applyExpressionFilters('%s=%r'%(settingName, valueExpr), + 'compiler', startPos=startPos) + + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLine) + try: + self._compiler.setCompilerSetting(settingName, valueExpr) + except: + sys.stderr.write('An error occurred while processing the following #compiler directive.\n') + sys.stderr.write('----------------------------------------------------------------------\n') + sys.stderr.write('%s\n' % self[startPos:endPos]) + sys.stderr.write('----------------------------------------------------------------------\n') + sys.stderr.write('Please check the syntax of these settings.\n\n') + raise + + + def eatCompilerSettings(self): + # filtered + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLine = self.findEOL() + self.getDirectiveStartToken() + self.advance(len('compiler-settings')) # to end of 'settings' + + keywords = self.getTargetVarsList() + self.getExpression() # gobble any garbage + + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLine) + + if 'reset' in keywords: + self._compiler._initializeSettings() + self.configureParser() + # @@TR: this implies a single-line #compiler-settings directive, and + # thus we should parse forward for an end directive. + # Subject to change in the future + return + startPos = self.pos() + settingsStr = self._eatToThisEndDirective('compiler-settings') + settingsStr = self._applyExpressionFilters(settingsStr, 'compilerSettings', + startPos=startPos) + try: + self._compiler.setCompilerSettings(keywords=keywords, settingsStr=settingsStr) + except: + sys.stderr.write('An error occurred while processing the following compiler settings.\n') + sys.stderr.write('----------------------------------------------------------------------\n') + sys.stderr.write('%s\n' % settingsStr.strip()) + sys.stderr.write('----------------------------------------------------------------------\n') + sys.stderr.write('Please check the syntax of these settings.\n\n') + raise + + def eatAttr(self): + # filtered + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLinePos = self.findEOL() + startPos = self.pos() + self.getDirectiveStartToken() + self.advance(len('attr')) + self.getWhiteSpace() + startPos = self.pos() + if self.matchCheetahVarStart(): + self.getCheetahVarStartToken() + attribName = self.getIdentifier() + self.getWhiteSpace() + self.getAssignmentOperator() + expr = self.getExpression() + expr = self._applyExpressionFilters(expr, 'attr', startPos=startPos) + self._compiler.addAttribute(attribName, expr) + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLinePos) + + def eatDecorator(self): + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLinePos = self.findEOL() + startPos = self.pos() + self.getDirectiveStartToken() + #self.advance() # eat @ + startPos = self.pos() + decoratorExpr = self.getExpression() + decoratorExpr = self._applyExpressionFilters(decoratorExpr, 'decorator', startPos=startPos) + self._compiler.addDecorator(decoratorExpr) + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLinePos) + self.getWhiteSpace() + + directiveName = self.matchDirective() + if not directiveName or directiveName not in ('def', 'block', 'closure', '@'): + raise ParseError( + self, msg='Expected #def, #block, #closure or another @decorator') + self.eatDirective() + + def eatDef(self): + # filtered + self._eatDefOrBlock('def') + + def eatBlock(self): + # filtered + startPos = self.pos() + methodName, rawSignature = self._eatDefOrBlock('block') + self._compiler._blockMetaData[methodName] = { + 'raw': rawSignature, + 'lineCol': self.getRowCol(startPos), + } + + def eatClosure(self): + # filtered + self._eatDefOrBlock('closure') + + def _eatDefOrBlock(self, directiveName): + # filtered + assert directiveName in ('def', 'block', 'closure') + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLinePos = self.findEOL() + startPos = self.pos() + self.getDirectiveStartToken() + self.advance(len(directiveName)) + self.getWhiteSpace() + if self.matchCheetahVarStart(): + self.getCheetahVarStartToken() + methodName = self.getIdentifier() + self.getWhiteSpace() + if self.peek() == '(': + argsList = self.getDefArgList() + self.advance() # past the closing ')' + if argsList and argsList[0][0] == 'self': + del argsList[0] + else: + argsList=[] + + def includeBlockMarkers(): + if self.setting('includeBlockMarkers'): + startMarker = self.setting('blockMarkerStart') + self._compiler.addStrConst(startMarker[0] + methodName + startMarker[1]) + + # @@TR: fix up filtering + self._applyExpressionFilters(self[startPos:self.pos()], 'def', startPos=startPos) + + if self.matchColonForSingleLineShortFormDirective(): + isNestedDef = (self.setting('allowNestedDefScopes') + and [name for name in self._openDirectivesStack if name=='def']) + self.getc() + rawSignature = self[startPos:endOfFirstLinePos] + self._eatSingleLineDef(directiveName=directiveName, + methodName=methodName, + argsList=argsList, + startPos=startPos, + endPos=endOfFirstLinePos) + if directiveName == 'def' and not isNestedDef: + #@@TR: must come before _eatRestOfDirectiveTag ... for some reason + self._compiler.closeDef() + elif directiveName == 'block': + includeBlockMarkers() + self._compiler.closeBlock() + elif directiveName == 'closure' or isNestedDef: + self._compiler.dedent() + + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLinePos) + else: + if self.peek()==':': + self.getc() + self.pushToOpenDirectivesStack(directiveName) + rawSignature = self[startPos:self.pos()] + self._eatMultiLineDef(directiveName=directiveName, + methodName=methodName, + argsList=argsList, + startPos=startPos, + isLineClearToStartToken=isLineClearToStartToken) + if directiveName == 'block': + includeBlockMarkers() + + return methodName, rawSignature + + def _eatMultiLineDef(self, directiveName, methodName, argsList, startPos, + isLineClearToStartToken=False): + # filtered in calling method + self.getExpression() # slurp up any garbage left at the end + signature = self[startPos:self.pos()] + endOfFirstLinePos = self.findEOL() + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLinePos) + signature = ' '.join([line.strip() for line in signature.splitlines()]) + parserComment = ('## CHEETAH: generated from ' + signature + + ' at line %s, col %s' % self.getRowCol(startPos) + + '.') + + isNestedDef = (self.setting('allowNestedDefScopes') + and len([name for name in self._openDirectivesStack if name=='def'])>1) + if directiveName=='block' or (directiveName=='def' and not isNestedDef): + self._compiler.startMethodDef(methodName, argsList, parserComment) + else: #closure + self._useSearchList_orig = self.setting('useSearchList') + self.setSetting('useSearchList', False) + self._compiler.addClosure(methodName, argsList, parserComment) + + return methodName + + def _eatSingleLineDef(self, directiveName, methodName, argsList, startPos, endPos): + # filtered in calling method + fullSignature = self[startPos:endPos] + parserComment = ('## Generated from ' + fullSignature + + ' at line %s, col %s' % self.getRowCol(startPos) + + '.') + isNestedDef = (self.setting('allowNestedDefScopes') + and [name for name in self._openDirectivesStack if name=='def']) + if directiveName=='block' or (directiveName=='def' and not isNestedDef): + self._compiler.startMethodDef(methodName, argsList, parserComment) + else: #closure + # @@TR: temporary hack of useSearchList + useSearchList_orig = self.setting('useSearchList') + self.setSetting('useSearchList', False) + self._compiler.addClosure(methodName, argsList, parserComment) + + self.getWhiteSpace(max=1) + self.parse(breakPoint=endPos) + if directiveName=='closure' or isNestedDef: # @@TR: temporary hack of useSearchList + self.setSetting('useSearchList', useSearchList_orig) + + def eatExtends(self): + # filtered + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLine = self.findEOL() + self.getDirectiveStartToken() + self.advance(len('extends')) + self.getWhiteSpace() + startPos = self.pos() + if self.setting('allowExpressionsInExtendsDirective'): + baseName = self.getExpression() + else: + baseName = self.getCommaSeparatedSymbols() + baseName = ', '.join(baseName) + + baseName = self._applyExpressionFilters(baseName, 'extends', startPos=startPos) + self._compiler.setBaseClass(baseName) # in compiler + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLine) + + def eatImplements(self): + # filtered + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLine = self.findEOL() + self.getDirectiveStartToken() + self.advance(len('implements')) + self.getWhiteSpace() + startPos = self.pos() + methodName = self.getIdentifier() + if not self.atEnd() and self.peek() == '(': + argsList = self.getDefArgList() + self.advance() # past the closing ')' + if argsList and argsList[0][0] == 'self': + del argsList[0] + else: + argsList=[] + + # @@TR: need to split up filtering of the methodname and the args + #methodName = self._applyExpressionFilters(methodName, 'implements', startPos=startPos) + self._applyExpressionFilters(self[startPos:self.pos()], 'implements', startPos=startPos) + + self._compiler.setMainMethodName(methodName) + self._compiler.setMainMethodArgs(argsList) + + self.getExpression() # throw away and unwanted crap that got added in + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLine) + + def eatSuper(self): + # filtered + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLine = self.findEOL() + self.getDirectiveStartToken() + self.advance(len('super')) + self.getWhiteSpace() + startPos = self.pos() + if not self.atEnd() and self.peek() == '(': + argsList = self.getDefArgList() + self.advance() # past the closing ')' + if argsList and argsList[0][0] == 'self': + del argsList[0] + else: + argsList=[] + + self._applyExpressionFilters(self[startPos:self.pos()], 'super', startPos=startPos) + + #parserComment = ('## CHEETAH: generated from ' + signature + + # ' at line %s, col %s' % self.getRowCol(startPos) + # + '.') + + self.getExpression() # throw away and unwanted crap that got added in + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLine) + self._compiler.addSuper(argsList) + + def eatSet(self): + # filtered + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLine = self.findEOL() + self.getDirectiveStartToken() + self.advance(3) + self.getWhiteSpace() + style = SET_LOCAL + if self.startswith('local'): + self.getIdentifier() + self.getWhiteSpace() + elif self.startswith('global'): + self.getIdentifier() + self.getWhiteSpace() + style = SET_GLOBAL + elif self.startswith('module'): + self.getIdentifier() + self.getWhiteSpace() + style = SET_MODULE + + startsWithDollar = self.matchCheetahVarStart() + startPos = self.pos() + LVALUE = self.getExpression(pyTokensToBreakAt=assignmentOps, useNameMapper=False).strip() + OP = self.getAssignmentOperator() + RVALUE = self.getExpression() + expr = LVALUE + ' ' + OP + ' ' + RVALUE.strip() + + expr = self._applyExpressionFilters(expr, 'set', startPos=startPos) + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLine) + + class Components: pass # used for 'set global' + exprComponents = Components() + exprComponents.LVALUE = LVALUE + exprComponents.OP = OP + exprComponents.RVALUE = RVALUE + self._compiler.addSet(expr, exprComponents, style) + + def eatSlurp(self): + if self.isLineClearToStartToken(): + self._compiler.handleWSBeforeDirective() + self._compiler.commitStrConst() + self.readToEOL(gobble=True) + + def eatEOLSlurpToken(self): + if self.isLineClearToStartToken(): + self._compiler.handleWSBeforeDirective() + self._compiler.commitStrConst() + self.readToEOL(gobble=True) + + def eatRaw(self): + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLinePos = self.findEOL() + self.getDirectiveStartToken() + self.advance(len('raw')) + self.getWhiteSpace() + if self.matchColonForSingleLineShortFormDirective(): + self.advance() # skip over : + self.getWhiteSpace(max=1) + rawBlock = self.readToEOL(gobble=False) + else: + if self.peek()==':': + self.advance() + self.getWhiteSpace() + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLinePos) + rawBlock = self._eatToThisEndDirective('raw') + self._compiler.addRawText(rawBlock) + + def eatInclude(self): + # filtered + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLinePos = self.findEOL() + self.getDirectiveStartToken() + self.advance(len('include')) + + self.getWhiteSpace() + includeFrom = 'file' + isRaw = False + if self.startswith('raw'): + self.advance(3) + isRaw=True + + self.getWhiteSpace() + if self.startswith('source'): + self.advance(len('source')) + includeFrom = 'str' + self.getWhiteSpace() + if not self.peek() == '=': + raise ParseError(self) + self.advance() + startPos = self.pos() + sourceExpr = self.getExpression() + sourceExpr = self._applyExpressionFilters(sourceExpr, 'include', startPos=startPos) + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLinePos) + self._compiler.addInclude(sourceExpr, includeFrom, isRaw) + + + def eatDefMacro(self): + # @@TR: not filtered yet + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLinePos = self.findEOL() + self.getDirectiveStartToken() + self.advance(len('defmacro')) + + self.getWhiteSpace() + if self.matchCheetahVarStart(): + self.getCheetahVarStartToken() + macroName = self.getIdentifier() + self.getWhiteSpace() + if self.peek() == '(': + argsList = self.getDefArgList(useNameMapper=False) + self.advance() # past the closing ')' + if argsList and argsList[0][0] == 'self': + del argsList[0] + else: + argsList=[] + + assert macroName not in self._directiveNamesAndParsers + argsList.insert(0, ('src', None)) + argsList.append(('parser', 'None')) + argsList.append(('macros', 'None')) + argsList.append(('compilerSettings', 'None')) + argsList.append(('isShortForm', 'None')) + argsList.append(('EOLCharsInShortForm', 'None')) + argsList.append(('startPos', 'None')) + argsList.append(('endPos', 'None')) + + if self.matchColonForSingleLineShortFormDirective(): + self.advance() # skip over : + self.getWhiteSpace(max=1) + macroSrc = self.readToEOL(gobble=False) + self.readToEOL(gobble=True) + else: + if self.peek()==':': + self.advance() + self.getWhiteSpace() + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLinePos) + macroSrc = self._eatToThisEndDirective('defmacro') + + #print argsList + normalizedMacroSrc = ''.join( + ['%def callMacro('+','.join([defv and '%s=%s'%(n, defv) or n + for n, defv in argsList]) + +')\n', + macroSrc, + '%end def']) + + + from Cheetah.Template import Template + templateAPIClass = self.setting('templateAPIClassForDefMacro', default=Template) + compilerSettings = self.setting('compilerSettingsForDefMacro', default={}) + searchListForMacros = self.setting('searchListForDefMacro', default=[]) + searchListForMacros = list(searchListForMacros) # copy to avoid mutation bugs + searchListForMacros.append({'macros': self._macros, + 'parser': self, + 'compilerSettings': self.settings(), + }) + + templateAPIClass._updateSettingsWithPreprocessTokens( + compilerSettings, placeholderToken='@', directiveToken='%') + macroTemplateClass = templateAPIClass.compile(source=normalizedMacroSrc, + compilerSettings=compilerSettings) + #print normalizedMacroSrc + #t = macroTemplateClass() + #print t.callMacro('src') + #print t.generatedClassCode() + + class MacroDetails: pass + macroDetails = MacroDetails() + macroDetails.macroSrc = macroSrc + macroDetails.argsList = argsList + macroDetails.template = macroTemplateClass(searchList=searchListForMacros) + + self._macroDetails[macroName] = macroDetails + self._macros[macroName] = macroDetails.template.callMacro + self._directiveNamesAndParsers[macroName] = self.eatMacroCall + + def eatMacroCall(self): + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLinePos = self.findEOL() + startPos = self.pos() + self.getDirectiveStartToken() + macroName = self.getIdentifier() + macro = self._macros[macroName] + if hasattr(macro, 'parse'): + return macro.parse(parser=self, startPos=startPos) + + if hasattr(macro, 'parseArgs'): + args = macro.parseArgs(parser=self, startPos=startPos) + else: + self.getWhiteSpace() + args = self.getExpression(useNameMapper=False, + pyTokensToBreakAt=[':']).strip() + + if self.matchColonForSingleLineShortFormDirective(): + isShortForm = True + self.advance() # skip over : + self.getWhiteSpace(max=1) + srcBlock = self.readToEOL(gobble=False) + EOLCharsInShortForm = self.readToEOL(gobble=True) + #self.readToEOL(gobble=False) + else: + isShortForm = False + if self.peek()==':': + self.advance() + self.getWhiteSpace() + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLinePos) + srcBlock = self._eatToThisEndDirective(macroName) + + + if hasattr(macro, 'convertArgStrToDict'): + kwArgs = macro.convertArgStrToDict(args, parser=self, startPos=startPos) + else: + def getArgs(*pargs, **kws): + return pargs, kws + exec('positionalArgs, kwArgs = getArgs(%(args)s)'%locals()) + + assert 'src' not in kwArgs + kwArgs['src'] = srcBlock + + if isinstance(macro, types.MethodType): + co = macro.im_func.func_code + elif (hasattr(macro, '__call__') + and hasattr(macro.__call__, 'im_func')): + co = macro.__call__.im_func.func_code + else: + co = macro.func_code + availableKwArgs = inspect.getargs(co)[0] + + if 'parser' in availableKwArgs: + kwArgs['parser'] = self + if 'macros' in availableKwArgs: + kwArgs['macros'] = self._macros + if 'compilerSettings' in availableKwArgs: + kwArgs['compilerSettings'] = self.settings() + if 'isShortForm' in availableKwArgs: + kwArgs['isShortForm'] = isShortForm + if isShortForm and 'EOLCharsInShortForm' in availableKwArgs: + kwArgs['EOLCharsInShortForm'] = EOLCharsInShortForm + + if 'startPos' in availableKwArgs: + kwArgs['startPos'] = startPos + if 'endPos' in availableKwArgs: + kwArgs['endPos'] = self.pos() + + srcFromMacroOutput = macro(**kwArgs) + + origParseSrc = self._src + origBreakPoint = self.breakPoint() + origPos = self.pos() + # add a comment to the output about the macro src that is being parsed + # or add a comment prefix to all the comments added by the compiler + self._src = srcFromMacroOutput + self.setPos(0) + self.setBreakPoint(len(srcFromMacroOutput)) + + self.parse(assertEmptyStack=False) + + self._src = origParseSrc + self.setBreakPoint(origBreakPoint) + self.setPos(origPos) + + + #self._compiler.addRawText('end') + + def eatCache(self): + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLinePos = self.findEOL() + lineCol = self.getRowCol() + self.getDirectiveStartToken() + self.advance(len('cache')) + + startPos = self.pos() + argList = self.getDefArgList(useNameMapper=True) + argList = self._applyExpressionFilters(argList, 'cache', startPos=startPos) + + def startCache(): + cacheInfo = self._compiler.genCacheInfoFromArgList(argList) + self._compiler.startCacheRegion(cacheInfo, lineCol) + + if self.matchColonForSingleLineShortFormDirective(): + self.advance() # skip over : + self.getWhiteSpace(max=1) + startCache() + self.parse(breakPoint=self.findEOL(gobble=True)) + self._compiler.endCacheRegion() + else: + if self.peek()==':': + self.advance() + self.getWhiteSpace() + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLinePos) + self.pushToOpenDirectivesStack('cache') + startCache() + + def eatCall(self): + # @@TR: need to enable single line version of this + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLinePos = self.findEOL() + lineCol = self.getRowCol() + self.getDirectiveStartToken() + self.advance(len('call')) + startPos = self.pos() + + useAutocallingOrig = self.setting('useAutocalling') + self.setSetting('useAutocalling', False) + self.getWhiteSpace() + if self.matchCheetahVarStart(): + functionName = self.getCheetahVar() + else: + functionName = self.getCheetahVar(plain=True, skipStartToken=True) + self.setSetting('useAutocalling', useAutocallingOrig) + # @@TR: fix up filtering + self._applyExpressionFilters(self[startPos:self.pos()], 'call', startPos=startPos) + + self.getWhiteSpace() + args = self.getExpression(pyTokensToBreakAt=[':']).strip() + if self.matchColonForSingleLineShortFormDirective(): + self.advance() # skip over : + self._compiler.startCallRegion(functionName, args, lineCol) + self.getWhiteSpace(max=1) + self.parse(breakPoint=self.findEOL(gobble=False)) + self._compiler.endCallRegion() + else: + if self.peek()==':': + self.advance() + self.getWhiteSpace() + self.pushToOpenDirectivesStack("call") + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLinePos) + self._compiler.startCallRegion(functionName, args, lineCol) + + def eatCallArg(self): + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLinePos = self.findEOL() + lineCol = self.getRowCol() + self.getDirectiveStartToken() + + self.advance(len('arg')) + startPos = self.pos() + self.getWhiteSpace() + argName = self.getIdentifier() + self.getWhiteSpace() + argName = self._applyExpressionFilters(argName, 'arg', startPos=startPos) + self._compiler.setCallArg(argName, lineCol) + if self.peek() == ':': + self.getc() + else: + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLinePos) + + def eatFilter(self): + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLinePos = self.findEOL() + + self.getDirectiveStartToken() + self.advance(len('filter')) + self.getWhiteSpace() + startPos = self.pos() + if self.matchCheetahVarStart(): + isKlass = True + theFilter = self.getExpression(pyTokensToBreakAt=[':']) + else: + isKlass = False + theFilter = self.getIdentifier() + self.getWhiteSpace() + theFilter = self._applyExpressionFilters(theFilter, 'filter', startPos=startPos) + + if self.matchColonForSingleLineShortFormDirective(): + self.advance() # skip over : + self.getWhiteSpace(max=1) + self._compiler.setFilter(theFilter, isKlass) + self.parse(breakPoint=self.findEOL(gobble=False)) + self._compiler.closeFilterBlock() + else: + if self.peek()==':': + self.advance() + self.getWhiteSpace() + self.pushToOpenDirectivesStack("filter") + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLinePos) + self._compiler.setFilter(theFilter, isKlass) + + def eatTransform(self): + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLinePos = self.findEOL() + + self.getDirectiveStartToken() + self.advance(len('transform')) + self.getWhiteSpace() + startPos = self.pos() + if self.matchCheetahVarStart(): + isKlass = True + transformer = self.getExpression(pyTokensToBreakAt=[':']) + else: + isKlass = False + transformer = self.getIdentifier() + self.getWhiteSpace() + transformer = self._applyExpressionFilters(transformer, 'transform', startPos=startPos) + + if self.peek()==':': + self.advance() + self.getWhiteSpace() + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLinePos) + self._compiler.setTransform(transformer, isKlass) + + + def eatErrorCatcher(self): + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLinePos = self.findEOL() + self.getDirectiveStartToken() + self.advance(len('errorCatcher')) + self.getWhiteSpace() + startPos = self.pos() + errorCatcherName = self.getIdentifier() + errorCatcherName = self._applyExpressionFilters( + errorCatcherName, 'errorcatcher', startPos=startPos) + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLinePos) + self._compiler.setErrorCatcher(errorCatcherName) + + def eatCapture(self): + # @@TR: this could be refactored to use the code in eatSimpleIndentingDirective + # filtered + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLinePos = self.findEOL() + lineCol = self.getRowCol() + + self.getDirectiveStartToken() + self.advance(len('capture')) + startPos = self.pos() + self.getWhiteSpace() + + expr = self.getExpression(pyTokensToBreakAt=[':']) + expr = self._applyExpressionFilters(expr, 'capture', startPos=startPos) + if self.matchColonForSingleLineShortFormDirective(): + self.advance() # skip over : + self._compiler.startCaptureRegion(assignTo=expr, lineCol=lineCol) + self.getWhiteSpace(max=1) + self.parse(breakPoint=self.findEOL(gobble=False)) + self._compiler.endCaptureRegion() + else: + if self.peek()==':': + self.advance() + self.getWhiteSpace() + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLinePos) + self.pushToOpenDirectivesStack("capture") + self._compiler.startCaptureRegion(assignTo=expr, lineCol=lineCol) + + + def eatIf(self): + # filtered + isLineClearToStartToken = self.isLineClearToStartToken() + endOfFirstLine = self.findEOL() + lineCol = self.getRowCol() + self.getDirectiveStartToken() + startPos = self.pos() + + expressionParts = self.getExpressionParts(pyTokensToBreakAt=[':']) + expr = ''.join(expressionParts).strip() + expr = self._applyExpressionFilters(expr, 'if', startPos=startPos) + + isTernaryExpr = ('then' in expressionParts and 'else' in expressionParts) + if isTernaryExpr: + conditionExpr = [] + trueExpr = [] + falseExpr = [] + currentExpr = conditionExpr + for part in expressionParts: + if part.strip()=='then': + currentExpr = trueExpr + elif part.strip()=='else': + currentExpr = falseExpr + else: + currentExpr.append(part) + + conditionExpr = ''.join(conditionExpr) + trueExpr = ''.join(trueExpr) + falseExpr = ''.join(falseExpr) + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLine) + self._compiler.addTernaryExpr(conditionExpr, trueExpr, falseExpr, lineCol=lineCol) + elif self.matchColonForSingleLineShortFormDirective(): + self.advance() # skip over : + self._compiler.addIf(expr, lineCol=lineCol) + self.getWhiteSpace(max=1) + self.parse(breakPoint=self.findEOL(gobble=True)) + self._compiler.commitStrConst() + self._compiler.dedent() + else: + if self.peek()==':': + self.advance() + self.getWhiteSpace() + self._eatRestOfDirectiveTag(isLineClearToStartToken, endOfFirstLine) + self.pushToOpenDirectivesStack('if') + self._compiler.addIf(expr, lineCol=lineCol) + + ## end directive handlers + def handleEndDef(self): + isNestedDef = (self.setting('allowNestedDefScopes') + and [name for name in self._openDirectivesStack if name=='def']) + if not isNestedDef: + self._compiler.closeDef() + else: + # @@TR: temporary hack of useSearchList + self.setSetting('useSearchList', self._useSearchList_orig) + self._compiler.commitStrConst() + self._compiler.dedent() + ### + + def pushToOpenDirectivesStack(self, directiveName): + assert directiveName in self._closeableDirectives + self._openDirectivesStack.append(directiveName) + + def popFromOpenDirectivesStack(self, directiveName): + if not self._openDirectivesStack: + raise ParseError(self, msg="#end found, but nothing to end") + + if self._openDirectivesStack[-1] == directiveName: + del self._openDirectivesStack[-1] + else: + raise ParseError(self, msg="#end %s found, expected #end %s" %( + directiveName, self._openDirectivesStack[-1])) + + def assertEmptyOpenDirectivesStack(self): + if self._openDirectivesStack: + errorMsg = ( + "Some #directives are missing their corresponding #end ___ tag: %s" %( + ', '.join(self._openDirectivesStack))) + raise ParseError(self, msg=errorMsg) + +################################################## +## Make an alias to export +Parser = _HighLevelParser === added file 'cheetah/Servlet.py' --- cheetah/Servlet.py 1970-01-01 00:00:00 +0000 +++ cheetah/Servlet.py 2010-10-19 19:11:10 +0000 @@ -0,0 +1,48 @@ +#!/usr/bin/env python +''' +Provides an abstract Servlet baseclass for Cheetah's Template class +''' + +import sys +import os.path + +class Servlet(object): + """ + This class is an abstract baseclass for Cheetah.Template.Template. + """ + + transaction = None + application = None + request = None + session = None + + def respond(self, trans=None): + raise NotImplementedError("""\ +couldn't find the template's main method. If you are using #extends +without #implements, try adding '#implements respond' to your template +definition.""") + + def sleep(self, transaction): + super(Servlet, self).sleep(transaction) + self.session = None + self.request = None + self._request = None + self.response = None + self.transaction = None + + def shutdown(self): + pass + + def serverSidePath(self, path=None, + normpath=os.path.normpath, + abspath=os.path.abspath + ): + + if path: + return normpath(abspath(path.replace("\\", '/'))) + elif hasattr(self, '_filePath') and self._filePath: + return normpath(abspath(self._filePath)) + else: + return None + +# vim: shiftwidth=4 tabstop=4 expandtab === added file 'cheetah/SettingsManager.py' --- cheetah/SettingsManager.py 1970-01-01 00:00:00 +0000 +++ cheetah/SettingsManager.py 2010-10-19 19:11:10 +0000 @@ -0,0 +1,284 @@ +import sys +import os.path +import copy as copyModule +from ConfigParser import ConfigParser +import re +from tokenize import Intnumber, Floatnumber, Number +import types +import time +from StringIO import StringIO # not cStringIO because of unicode support +import imp # used by SettingsManager.updateSettingsFromPySrcFile() + + +numberRE = re.compile(Number) +complexNumberRE = re.compile('[\(]*' +Number + r'[ \t]*\+[ \t]*' + Number + '[\)]*') + +################################################## +## FUNCTIONS ## + +def mergeNestedDictionaries(dict1, dict2, copy=False, deepcopy=False): + """Recursively merge the values of dict2 into dict1. + + This little function is very handy for selectively overriding settings in a + settings dictionary that has a nested structure. + """ + + if copy: + dict1 = copyModule.copy(dict1) + elif deepcopy: + dict1 = copyModule.deepcopy(dict1) + + for key, val in dict2.iteritems(): + if key in dict1 and isinstance(val, dict) and isinstance(dict1[key], dict): + dict1[key] = mergeNestedDictionaries(dict1[key], val) + else: + dict1[key] = val + return dict1 + +def stringIsNumber(S): + """Return True if theString represents a Python number, False otherwise. + This also works for complex numbers and numbers with +/- in front.""" + + S = S.strip() + + if S[0] in '-+' and len(S) > 1: + S = S[1:].strip() + + match = complexNumberRE.match(S) + if not match: + match = numberRE.match(S) + if not match or (match.end() != len(S)): + return False + else: + return True + +def convStringToNum(theString): + """Convert a string representation of a Python number to the Python version""" + + if not stringIsNumber(theString): + raise Error(theString + ' cannot be converted to a Python number') + return eval(theString, {}, {}) + + +class Error(Exception): + pass + +class NoDefault(object): + pass + +class ConfigParserCaseSensitive(ConfigParser): + """A case sensitive version of the standard Python ConfigParser.""" + + def optionxform(self, optionstr): + """Don't change the case as is done in the default implemenation.""" + return optionstr + +class _SettingsCollector(object): + """An abstract base class that provides the methods SettingsManager uses to + collect settings from config files and strings. + + This class only collects settings it doesn't modify the _settings dictionary + of SettingsManager instances in any way. + """ + + _ConfigParserClass = ConfigParserCaseSensitive + + def readSettingsFromModule(self, mod, ignoreUnderscored=True): + """Returns all settings from a Python module. + """ + S = {} + attrs = vars(mod) + for k, v in attrs.iteritems(): + if (ignoreUnderscored and k.startswith('_')): + continue + else: + S[k] = v + return S + + def readSettingsFromPySrcStr(self, theString): + """Return a dictionary of the settings in a Python src string.""" + + globalsDict = {'True': (1==1), + 'False': (0==1), + } + newSettings = {'self':self} + exec((theString+os.linesep), globalsDict, newSettings) + del newSettings['self'] + module = types.ModuleType('temp_settings_module') + module.__dict__.update(newSettings) + return self.readSettingsFromModule(module) + + def readSettingsFromConfigFileObj(self, inFile, convert=True): + """Return the settings from a config file that uses the syntax accepted by + Python's standard ConfigParser module (like Windows .ini files). + + NOTE: + this method maintains case unlike the ConfigParser module, unless this + class was initialized with the 'caseSensitive' keyword set to False. + + All setting values are initially parsed as strings. However, If the + 'convert' arg is True this method will do the following value + conversions: + + * all Python numeric literals will be coverted from string to number + + * The string 'None' will be converted to the Python value None + + * The string 'True' will be converted to a Python truth value + + * The string 'False' will be converted to a Python false value + + * Any string starting with 'python:' will be treated as a Python literal + or expression that needs to be eval'd. This approach is useful for + declaring lists and dictionaries. + + If a config section titled 'Globals' is present the options defined + under it will be treated as top-level settings. + """ + + p = self._ConfigParserClass() + p.readfp(inFile) + sects = p.sections() + newSettings = {} + + sects = p.sections() + newSettings = {} + + for s in sects: + newSettings[s] = {} + for o in p.options(s): + if o != '__name__': + newSettings[s][o] = p.get(s, o) + + ## loop through new settings -> deal with global settings, numbers, + ## booleans and None ++ also deal with 'importSettings' commands + + for sect, subDict in newSettings.items(): + for key, val in subDict.items(): + if convert: + if val.lower().startswith('python:'): + subDict[key] = eval(val[7:], {}, {}) + if val.lower() == 'none': + subDict[key] = None + if val.lower() == 'true': + subDict[key] = True + if val.lower() == 'false': + subDict[key] = False + if stringIsNumber(val): + subDict[key] = convStringToNum(val) + + ## now deal with any 'importSettings' commands + if key.lower() == 'importsettings': + if val.find(';') < 0: + importedSettings = self.readSettingsFromPySrcFile(val) + else: + path = val.split(';')[0] + rest = ''.join(val.split(';')[1:]).strip() + parentDict = self.readSettingsFromPySrcFile(path) + importedSettings = eval('parentDict["' + rest + '"]') + + subDict.update(mergeNestedDictionaries(subDict, + importedSettings)) + + if sect.lower() == 'globals': + newSettings.update(newSettings[sect]) + del newSettings[sect] + + return newSettings + + +class SettingsManager(_SettingsCollector): + """A mixin class that provides facilities for managing application settings. + + SettingsManager is designed to work well with nested settings dictionaries + of any depth. + """ + + def __init__(self): + super(SettingsManager, self).__init__() + self._settings = {} + self._initializeSettings() + + def _defaultSettings(self): + return {} + + def _initializeSettings(self): + """A hook that allows for complex setting initialization sequences that + involve references to 'self' or other settings. For example: + self._settings['myCalcVal'] = self._settings['someVal'] * 15 + This method should be called by the class' __init__() method when needed. + The dummy implementation should be reimplemented by subclasses. + """ + + pass + + ## core post startup methods + + def setting(self, name, default=NoDefault): + """Get a setting from self._settings, with or without a default value.""" + + if default is NoDefault: + return self._settings[name] + else: + return self._settings.get(name, default) + + + def hasSetting(self, key): + """True/False""" + return key in self._settings + + def setSetting(self, name, value): + """Set a setting in self._settings.""" + self._settings[name] = value + + def settings(self): + """Return a reference to the settings dictionary""" + return self._settings + + def copySettings(self): + """Returns a shallow copy of the settings dictionary""" + return copyModule.copy(self._settings) + + def deepcopySettings(self): + """Returns a deep copy of the settings dictionary""" + return copyModule.deepcopy(self._settings) + + def updateSettings(self, newSettings, merge=True): + """Update the settings with a selective merge or a complete overwrite.""" + + if merge: + mergeNestedDictionaries(self._settings, newSettings) + else: + self._settings.update(newSettings) + + + ## source specific update methods + + def updateSettingsFromPySrcStr(self, theString, merge=True): + """Update the settings from a code in a Python src string.""" + + newSettings = self.readSettingsFromPySrcStr(theString) + self.updateSettings(newSettings, + merge=newSettings.get('mergeSettings', merge) ) + + + def updateSettingsFromConfigFileObj(self, inFile, convert=True, merge=True): + """See the docstring for .updateSettingsFromConfigFile() + + The caller of this method is responsible for closing the inFile file + object.""" + + newSettings = self.readSettingsFromConfigFileObj(inFile, convert=convert) + self.updateSettings(newSettings, + merge=newSettings.get('mergeSettings', merge)) + + def updateSettingsFromConfigStr(self, configStr, convert=True, merge=True): + """See the docstring for .updateSettingsFromConfigFile() + """ + + configStr = '[globals]\n' + configStr + inFile = StringIO(configStr) + newSettings = self.readSettingsFromConfigFileObj(inFile, convert=convert) + self.updateSettings(newSettings, + merge=newSettings.get('mergeSettings', merge)) + === added file 'cheetah/SourceReader.py' --- cheetah/SourceReader.py 1970-01-01 00:00:00 +0000 +++ cheetah/SourceReader.py 2010-10-19 19:11:10 +0000 @@ -0,0 +1,267 @@ +"""SourceReader class for Cheetah's Parser and CodeGenerator +""" +import re +import sys + +EOLre = re.compile(r'[ \f\t]*(?:\r\n|\r|\n)') +EOLZre = re.compile(r'(?:\r\n|\r|\n|\Z)') +ENCODINGsearch = re.compile("coding[=:]\s*([-\w.]+)").search + +class Error(Exception): + pass + +class SourceReader(object): + def __init__(self, src, filename=None, breakPoint=None, encoding=None): + self._src = src + self._filename = filename + self._srcLen = len(src) + if breakPoint == None: + self._breakPoint = self._srcLen + else: + self.setBreakPoint(breakPoint) + self._pos = 0 + self._bookmarks = {} + self._posTobookmarkMap = {} + + ## collect some meta-information + self._EOLs = [] + pos = 0 + while pos < len(self): + EOLmatch = EOLZre.search(src, pos) + self._EOLs.append(EOLmatch.start()) + pos = EOLmatch.end() + + self._BOLs = [] + for pos in self._EOLs: + BOLpos = self.findBOL(pos) + self._BOLs.append(BOLpos) + + def src(self): + return self._src + + def filename(self): + return self._filename + + def __len__(self): + return self._breakPoint + + def __getitem__(self, i): + if not isinstance(i, int): + self.checkPos(i.stop) + else: + self.checkPos(i) + return self._src[i] + + def __getslice__(self, i, j): + i = max(i, 0); j = max(j, 0) + return self._src[i:j] + + def splitlines(self): + if not hasattr(self, '_srcLines'): + self._srcLines = self._src.splitlines() + return self._srcLines + + def lineNum(self, pos=None): + if pos == None: + pos = self._pos + + for i in range(len(self._BOLs)): + if pos >= self._BOLs[i] and pos <= self._EOLs[i]: + return i + + def getRowCol(self, pos=None): + if pos == None: + pos = self._pos + lineNum = self.lineNum(pos) + BOL, EOL = self._BOLs[lineNum], self._EOLs[lineNum] + return lineNum+1, pos-BOL+1 + + def getRowColLine(self, pos=None): + if pos == None: + pos = self._pos + row, col = self.getRowCol(pos) + return row, col, self.splitlines()[row-1] + + def getLine(self, pos): + if pos == None: + pos = self._pos + lineNum = self.lineNum(pos) + return self.splitlines()[lineNum] + + def pos(self): + return self._pos + + def setPos(self, pos): + self.checkPos(pos) + self._pos = pos + + + def validPos(self, pos): + return pos <= self._breakPoint and pos >=0 + + def checkPos(self, pos): + if not pos <= self._breakPoint: + raise Error("pos (" + str(pos) + ") is invalid: beyond the stream's end (" + + str(self._breakPoint-1) + ")" ) + elif not pos >=0: + raise Error("pos (" + str(pos) + ") is invalid: less than 0" ) + + def breakPoint(self): + return self._breakPoint + + def setBreakPoint(self, pos): + if pos > self._srcLen: + raise Error("New breakpoint (" + str(pos) + + ") is invalid: beyond the end of stream's source string (" + + str(self._srcLen) + ")" ) + elif not pos >= 0: + raise Error("New breakpoint (" + str(pos) + ") is invalid: less than 0" ) + + self._breakPoint = pos + + def setBookmark(self, name): + self._bookmarks[name] = self._pos + self._posTobookmarkMap[self._pos] = name + + def hasBookmark(self, name): + return name in self._bookmarks + + def gotoBookmark(self, name): + if not self.hasBookmark(name): + raise Error("Invalid bookmark (" + name + ") is invalid: does not exist") + pos = self._bookmarks[name] + if not self.validPos(pos): + raise Error("Invalid bookmark (" + name + ', '+ + str(pos) + ") is invalid: pos is out of range" ) + self._pos = pos + + def atEnd(self): + return self._pos >= self._breakPoint + + def atStart(self): + return self._pos == 0 + + def peek(self, offset=0): + self.checkPos(self._pos+offset) + pos = self._pos + offset + return self._src[pos] + + def getc(self): + pos = self._pos + if self.validPos(pos+1): + self._pos += 1 + return self._src[pos] + + def ungetc(self, c=None): + if not self.atStart(): + raise Error('Already at beginning of stream') + + self._pos -= 1 + if not c==None: + self._src[self._pos] = c + + def advance(self, offset=1): + self.checkPos(self._pos + offset) + self._pos += offset + + def rev(self, offset=1): + self.checkPos(self._pos - offset) + self._pos -= offset + + def read(self, offset): + self.checkPos(self._pos + offset) + start = self._pos + self._pos += offset + return self._src[start:self._pos] + + def readTo(self, to, start=None): + self.checkPos(to) + if start == None: + start = self._pos + self._pos = to + return self._src[start:to] + + + def readToEOL(self, start=None, gobble=True): + EOLmatch = EOLZre.search(self.src(), self.pos()) + if gobble: + pos = EOLmatch.end() + else: + pos = EOLmatch.start() + return self.readTo(to=pos, start=start) + + + def find(self, it, pos=None): + if pos == None: + pos = self._pos + return self._src.find(it, pos ) + + def startswith(self, it, pos=None): + if self.find(it, pos) == self.pos(): + return True + else: + return False + + def rfind(self, it, pos): + if pos == None: + pos = self._pos + return self._src.rfind(it, pos) + + def findBOL(self, pos=None): + if pos == None: + pos = self._pos + src = self.src() + return max(src.rfind('\n', 0, pos)+1, src.rfind('\r', 0, pos)+1, 0) + + def findEOL(self, pos=None, gobble=False): + if pos == None: + pos = self._pos + + match = EOLZre.search(self.src(), pos) + if gobble: + return match.end() + else: + return match.start() + + def isLineClearToPos(self, pos=None): + if pos == None: + pos = self.pos() + self.checkPos(pos) + src = self.src() + BOL = self.findBOL() + return BOL == pos or src[BOL:pos].isspace() + + def matches(self, strOrRE): + if isinstance(strOrRE, (str, unicode)): + return self.startswith(strOrRE, pos=self.pos()) + else: # assume an re object + return strOrRE.match(self.src(), self.pos()) + + def matchWhiteSpace(self, WSchars=' \f\t'): + return (not self.atEnd()) and self.peek() in WSchars + + def getWhiteSpace(self, max=None, WSchars=' \f\t'): + if not self.matchWhiteSpace(WSchars): + return '' + start = self.pos() + breakPoint = self.breakPoint() + if max is not None: + breakPoint = min(breakPoint, self.pos()+max) + while self.pos() < breakPoint: + self.advance() + if not self.matchWhiteSpace(WSchars): + break + return self.src()[start:self.pos()] + + def matchNonWhiteSpace(self, WSchars=' \f\t\n\r'): + return self.atEnd() or not self.peek() in WSchars + + def getNonWhiteSpace(self, WSchars=' \f\t\n\r'): + if not self.matchNonWhiteSpace(WSchars): + return '' + start = self.pos() + while self.pos() < self.breakPoint(): + self.advance() + if not self.matchNonWhiteSpace(WSchars): + break + return self.src()[start:self.pos()] === added file 'cheetah/Template.py' --- cheetah/Template.py 1970-01-01 00:00:00 +0000 +++ cheetah/Template.py 2010-10-19 19:11:10 +0000 @@ -0,0 +1,1941 @@ +''' +Provides the core API for Cheetah. + +See the docstring in the Template class and the Users' Guide for more information +''' + +################################################################################ +## DEPENDENCIES +import sys # used in the error handling code +import re # used to define the internal delims regex +import logging +import string +import os.path +import time # used in the cache refresh code +from random import randrange +import imp +import inspect +import StringIO +import traceback +import pprint +import cgi # Used by .webInput() if the template is a CGI script. +import types + +try: + from threading import Lock +except ImportError: + class Lock: + def acquire(self): + pass + def release(self): + pass + +filetype = None + +if isinstance(sys.version_info, tuple): + # Python 2.xx + filetype = types.FileType + def createMethod(func, cls): + return types.MethodType(func, None, cls) +else: + import io + filetype = io.IOBase + def createMethod(func, cls): + return types.MethodType(func, cls) + + + +from Cheetah.Version import convertVersionStringToTuple, MinCompatibleVersionTuple +from Cheetah.Version import MinCompatibleVersion +# Base classes for Template +from Cheetah.Servlet import Servlet +# More intra-package imports ... +from Cheetah.Parser import ParseError, SourceReader +from Cheetah.Compiler import Compiler, DEFAULT_COMPILER_SETTINGS +from Cheetah import ErrorCatchers # for placeholder tags +from Cheetah import Filters # the output filters +from Cheetah.convertTmplPathToModuleName import convertTmplPathToModuleName + +from Cheetah.Utils.Misc import checkKeywords # Used in Template.__init__ +from Cheetah.Utils.Indenter import Indenter # Used in Template.__init__ and for + # placeholders +from Cheetah.NameMapper import NotFound, valueFromSearchList +from Cheetah.CacheStore import MemoryCacheStore, MemcachedCacheStore +from Cheetah.CacheRegion import CacheRegion +from Cheetah.Utils.WebInputMixin import _Converter, _lookup, NonNumericInputError + +from Cheetah.Unspecified import Unspecified + +# Decide whether to use the file modification time in file's cache key +__checkFileMtime = True +def checkFileMtime(value): + globals()['__checkFileMtime'] = value + +class Error(Exception): + pass +class PreprocessError(Error): + pass + +def hashList(l): + hashedList = [] + for v in l: + if isinstance(v, dict): + v = hashDict(v) + elif isinstance(v, list): + v = hashList(v) + hashedList.append(v) + return hash(tuple(hashedList)) + +def hashDict(d): + items = sorted(d.items()) + hashedList = [] + for k, v in items: + if isinstance(v, dict): + v = hashDict(v) + elif isinstance(v, list): + v = hashList(v) + hashedList.append((k, v)) + return hash(tuple(hashedList)) + + +################################################################################ +## MODULE GLOBALS AND CONSTANTS + +def _genUniqueModuleName(baseModuleName): + """The calling code is responsible for concurrency locking. + """ + if baseModuleName not in sys.modules: + finalName = baseModuleName + else: + finalName = ('cheetah_%s_%s_%s'%(baseModuleName, + str(time.time()).replace('.', '_'), + str(randrange(10000, 99999)))) + return finalName + +# Cache of a cgi.FieldStorage() instance, maintained by .webInput(). +# This is only relavent to templates used as CGI scripts. +_formUsedByWebInput = None + +def updateLinecache(filename, src): + import linecache + size = len(src) + mtime = time.time() + lines = src.splitlines() + fullname = filename + linecache.cache[filename] = size, mtime, lines, fullname + +class CompileCacheItem(object): + pass + +class TemplatePreprocessor(object): + ''' + This is used with the preprocessors argument to Template.compile(). + + See the docstring for Template.compile + + ** Preprocessors are an advanced topic ** + ''' + + def __init__(self, settings): + self._settings = settings + + def preprocess(self, source, file): + """Create an intermediate template and return the source code + it outputs + """ + settings = self._settings + if not source: # @@TR: this needs improving + if isinstance(file, (str, unicode)): # it's a filename. + f = open(file) + source = f.read() + f.close() + elif hasattr(file, 'read'): + source = file.read() + file = None + + templateAPIClass = settings.templateAPIClass + possibleKwArgs = [ + arg for arg in + inspect.getargs(templateAPIClass.compile.im_func.func_code)[0] + if arg not in ('klass', 'source', 'file',)] + + compileKwArgs = {} + for arg in possibleKwArgs: + if hasattr(settings, arg): + compileKwArgs[arg] = getattr(settings, arg) + + tmplClass = templateAPIClass.compile(source=source, file=file, **compileKwArgs) + tmplInstance = tmplClass(**settings.templateInitArgs) + outputSource = settings.outputTransformer(tmplInstance) + outputFile = None + return outputSource, outputFile + +class Template(Servlet): + ''' + This class provides a) methods used by templates at runtime and b) + methods for compiling Cheetah source code into template classes. + + This documentation assumes you already know Python and the basics of object + oriented programming. If you don't know Python, see the sections of the + Cheetah Users' Guide for non-programmers. It also assumes you have read + about Cheetah's syntax in the Users' Guide. + + The following explains how to use Cheetah from within Python programs or via + the interpreter. If you statically compile your templates on the command + line using the 'cheetah' script, this is not relevant to you. Statically + compiled Cheetah template modules/classes (e.g. myTemplate.py: + MyTemplateClasss) are just like any other Python module or class. Also note, + most Python web frameworks (Webware, Aquarium, mod_python, Turbogears, + CherryPy, Quixote, etc.) provide plugins that handle Cheetah compilation for + you. + + There are several possible usage patterns: + 1) tclass = Template.compile(src) + t1 = tclass() # or tclass(namespaces=[namespace,...]) + t2 = tclass() # or tclass(namespaces=[namespace2,...]) + outputStr = str(t1) # or outputStr = t1.aMethodYouDefined() + + Template.compile provides a rich and very flexible API via its + optional arguments so there are many possible variations of this + pattern. One example is: + tclass = Template.compile('hello $name from $caller', baseclass=dict) + print tclass(name='world', caller='me') + See the Template.compile() docstring for more details. + + 2) tmplInstance = Template(src) + # or Template(src, namespaces=[namespace,...]) + outputStr = str(tmplInstance) # or outputStr = tmplInstance.aMethodYouDefined(...args...) + + Notes on the usage patterns: + + usage pattern 1) + This is the most flexible, but it is slightly more verbose unless you + write a wrapper function to hide the plumbing. Under the hood, all + other usage patterns are based on this approach. Templates compiled + this way can #extend (subclass) any Python baseclass: old-style or + new-style (based on object or a builtin type). + + usage pattern 2) + This was Cheetah's original usage pattern. It returns an instance, + but you can still access the generated class via + tmplInstance.__class__. If you want to use several different + namespace 'searchLists' with a single template source definition, + you're better off with Template.compile (1). + + Limitations (use pattern 1 instead): + - Templates compiled this way can only #extend subclasses of the + new-style 'object' baseclass. Cheetah.Template is a subclass of + 'object'. You also can not #extend dict, list, or other builtin + types. + - If your template baseclass' __init__ constructor expects args there + is currently no way to pass them in. + + If you need to subclass a dynamically compiled Cheetah class, do something like this: + from Cheetah.Template import Template + T1 = Template.compile('$meth1 #def meth1: this is meth1 in T1') + T2 = Template.compile('#implements meth1\nthis is meth1 redefined in T2', baseclass=T1) + print T1, T1() + print T2, T2() + + + Note about class and instance attribute names: + Attributes used by Cheetah have a special prefix to avoid confusion with + the attributes of the templates themselves or those of template + baseclasses. + + Class attributes which are used in class methods look like this: + klass._CHEETAH_useCompilationCache (_CHEETAH_xxx) + + Instance attributes look like this: + klass._CHEETAH__globalSetVars (_CHEETAH__xxx with 2 underscores) + ''' + + # this is used by ._addCheetahPlumbingCodeToClass() + _CHEETAH_requiredCheetahMethods = ( + '_initCheetahInstance', + 'searchList', + 'errorCatcher', + 'getVar', + 'varExists', + 'getFileContents', + 'i18n', + 'runAsMainProgram', + 'respond', + 'shutdown', + 'webInput', + 'serverSidePath', + 'generatedClassCode', + 'generatedModuleCode', + + '_getCacheStore', + '_getCacheStoreIdPrefix', + '_createCacheRegion', + 'getCacheRegion', + 'getCacheRegions', + 'refreshCache', + + '_handleCheetahInclude', + '_getTemplateAPIClassForIncludeDirectiveCompilation', + ) + _CHEETAH_requiredCheetahClassMethods = ('subclass',) + _CHEETAH_requiredCheetahClassAttributes = ('cacheRegionClass', 'cacheStore', + 'cacheStoreIdPrefix', 'cacheStoreClass') + + ## the following are used by .compile(). Most are documented in its docstring. + _CHEETAH_cacheModuleFilesForTracebacks = False + _CHEETAH_cacheDirForModuleFiles = None # change to a dirname + + _CHEETAH_compileCache = dict() # cache store for compiled code and classes + # To do something other than simple in-memory caching you can create an + # alternative cache store. It just needs to support the basics of Python's + # mapping/dict protocol. E.g.: + # class AdvCachingTemplate(Template): + # _CHEETAH_compileCache = MemoryOrFileCache() + _CHEETAH_compileLock = Lock() # used to prevent race conditions + _CHEETAH_defaultMainMethodName = None + _CHEETAH_compilerSettings = None + _CHEETAH_compilerClass = Compiler + _CHEETAH_compilerInstance = None + _CHEETAH_cacheCompilationResults = True + _CHEETAH_useCompilationCache = True + _CHEETAH_keepRefToGeneratedCode = True + _CHEETAH_defaultBaseclassForTemplates = None + _CHEETAH_defaultClassNameForTemplates = None + # defaults to DEFAULT_COMPILER_SETTINGS['mainMethodName']: + _CHEETAH_defaultMainMethodNameForTemplates = None + _CHEETAH_defaultModuleNameForTemplates = 'DynamicallyCompiledCheetahTemplate' + _CHEETAH_defaultModuleGlobalsForTemplates = None + _CHEETAH_preprocessors = None + _CHEETAH_defaultPreprocessorClass = TemplatePreprocessor + + ## The following attributes are used by instance methods: + _CHEETAH_generatedModuleCode = None + NonNumericInputError = NonNumericInputError + _CHEETAH_cacheRegionClass = CacheRegion + _CHEETAH_cacheStoreClass = MemoryCacheStore + #_CHEETAH_cacheStoreClass = MemcachedCacheStore + _CHEETAH_cacheStore = None + _CHEETAH_cacheStoreIdPrefix = None + + @classmethod + def _getCompilerClass(klass, source=None, file=None): + return klass._CHEETAH_compilerClass + + @classmethod + def _getCompilerSettings(klass, source=None, file=None): + return klass._CHEETAH_compilerSettings + + @classmethod + def compile(klass, source=None, file=None, + returnAClass=True, + + compilerSettings=Unspecified, + compilerClass=Unspecified, + moduleName=None, + className=Unspecified, + mainMethodName=Unspecified, + baseclass=Unspecified, + moduleGlobals=Unspecified, + cacheCompilationResults=Unspecified, + useCache=Unspecified, + preprocessors=Unspecified, + cacheModuleFilesForTracebacks=Unspecified, + cacheDirForModuleFiles=Unspecified, + commandlineopts=None, + keepRefToGeneratedCode=Unspecified, + ): + + """ + The core API for compiling Cheetah source code into template classes. + + This class method compiles Cheetah source code and returns a python + class. You then create template instances using that class. All + Cheetah's other compilation API's use this method under the hood. + + Internally, this method a) parses the Cheetah source code and generates + Python code defining a module with a single class in it, b) dynamically + creates a module object with a unique name, c) execs the generated code + in that module's namespace then inserts the module into sys.modules, and + d) returns a reference to the generated class. If you want to get the + generated python source code instead, pass the argument + returnAClass=False. + + It caches generated code and classes. See the descriptions of the + arguments'cacheCompilationResults' and 'useCache' for details. This + doesn't mean that templates will automatically recompile themselves when + the source file changes. Rather, if you call Template.compile(src) or + Template.compile(file=path) repeatedly it will attempt to return a + cached class definition instead of recompiling. + + Hooks are provided template source preprocessing. See the notes on the + 'preprocessors' arg. + + If you are an advanced user and need to customize the way Cheetah parses + source code or outputs Python code, you should check out the + compilerSettings argument. + + Arguments: + You must provide either a 'source' or 'file' arg, but not both: + - source (string or None) + - file (string path, file-like object, or None) + + The rest of the arguments are strictly optional. All but the first + have defaults in attributes of the Template class which can be + overridden in subclasses of this class. Working with most of these is + an advanced topic. + + - returnAClass=True + If false, return the generated module code rather than a class. + + - compilerSettings (a dict) + Default: Template._CHEETAH_compilerSettings=None + + a dictionary of settings to override those defined in + DEFAULT_COMPILER_SETTINGS. These can also be overridden in your + template source code with the #compiler or #compiler-settings + directives. + + - compilerClass (a class) + Default: Template._CHEETAH_compilerClass=Cheetah.Compiler.Compiler + + a subclass of Cheetah.Compiler.Compiler. Mucking with this is a + very advanced topic. + + - moduleName (a string) + Default: + Template._CHEETAH_defaultModuleNameForTemplates + ='DynamicallyCompiledCheetahTemplate' + + What to name the generated Python module. If the provided value is + None and a file arg was given, the moduleName is created from the + file path. In all cases if the moduleName provided is already in + sys.modules it is passed through a filter that generates a unique + variant of the name. + + + - className (a string) + Default: Template._CHEETAH_defaultClassNameForTemplates=None + + What to name the generated Python class. If the provided value is + None, the moduleName is use as the class name. + + - mainMethodName (a string) + Default: + Template._CHEETAH_defaultMainMethodNameForTemplates + =None (and thus DEFAULT_COMPILER_SETTINGS['mainMethodName']) + + What to name the main output generating method in the compiled + template class. + + - baseclass (a string or a class) + Default: Template._CHEETAH_defaultBaseclassForTemplates=None + + Specifies the baseclass for the template without manually + including an #extends directive in the source. The #extends + directive trumps this arg. + + If the provided value is a string you must make sure that a class + reference by that name is available to your template, either by + using an #import directive or by providing it in the arg + 'moduleGlobals'. + + If the provided value is a class, Cheetah will handle all the + details for you. + + - moduleGlobals (a dict) + Default: Template._CHEETAH_defaultModuleGlobalsForTemplates=None + + A dict of vars that will be added to the global namespace of the + module the generated code is executed in, prior to the execution + of that code. This should be Python values, not code strings! + + - cacheCompilationResults (True/False) + Default: Template._CHEETAH_cacheCompilationResults=True + + Tells Cheetah to cache the generated code and classes so that they + can be reused if Template.compile() is called multiple times with + the same source and options. + + - useCache (True/False) + Default: Template._CHEETAH_useCompilationCache=True + + Should the compilation cache be used? If True and a previous + compilation created a cached template class with the same source + code, compiler settings and other options, the cached template + class will be returned. + + - cacheModuleFilesForTracebacks (True/False) + Default: Template._CHEETAH_cacheModuleFilesForTracebacks=False + + In earlier versions of Cheetah tracebacks from exceptions that + were raised inside dynamically compiled Cheetah templates were + opaque because Python didn't have access to a python source file + to use in the traceback: + + File "xxxx.py", line 192, in getTextiledContent + content = str(template(searchList=searchList)) + File "cheetah_yyyy.py", line 202, in __str__ + File "cheetah_yyyy.py", line 187, in respond + File "cheetah_yyyy.py", line 139, in writeBody + ZeroDivisionError: integer division or modulo by zero + + It is now possible to keep those files in a cache dir and allow + Python to include the actual source lines in tracebacks and makes + them much easier to understand: + + File "xxxx.py", line 192, in getTextiledContent + content = str(template(searchList=searchList)) + File "/tmp/CheetahCacheDir/cheetah_yyyy.py", line 202, in __str__ + def __str__(self): return self.respond() + File "/tmp/CheetahCacheDir/cheetah_yyyy.py", line 187, in respond + self.writeBody(trans=trans) + File "/tmp/CheetahCacheDir/cheetah_yyyy.py", line 139, in writeBody + __v = 0/0 # $(0/0) + ZeroDivisionError: integer division or modulo by zero + + - cacheDirForModuleFiles (a string representing a dir path) + Default: Template._CHEETAH_cacheDirForModuleFiles=None + + See notes on cacheModuleFilesForTracebacks. + + - preprocessors + Default: Template._CHEETAH_preprocessors=None + + ** THIS IS A VERY ADVANCED TOPIC ** + + These are used to transform the source code prior to compilation. + They provide a way to use Cheetah as a code generator for Cheetah + code. In other words, you use one Cheetah template to output the + source code for another Cheetah template. + + The major expected use cases are: + + a) 'compile-time caching' aka 'partial template binding', + wherein an intermediate Cheetah template is used to output + the source for the final Cheetah template. The intermediate + template is a mix of a modified Cheetah syntax (the + 'preprocess syntax') and standard Cheetah syntax. The + preprocessor syntax is executed at compile time and outputs + Cheetah code which is then compiled in turn. This approach + allows one to completely soft-code all the elements in the + template which are subject to change yet have it compile to + extremely efficient Python code with everything but the + elements that must be variable at runtime (per browser + request, etc.) compiled as static strings. Examples of this + usage pattern will be added to the Cheetah Users' Guide. + + The'preprocess syntax' is just Cheetah's standard one with + alternatives for the $ and # tokens: + + e.g. '@' and '%' for code like this + @aPreprocessVar $aRuntimeVar + %if aCompileTimeCondition then yyy else zzz + %% preprocessor comment + + #if aRunTimeCondition then aaa else bbb + ## normal comment + $aRuntimeVar + + b) adding #import and #extends directives dynamically based on + the source + + If preprocessors are provided, Cheetah pipes the source code + through each one in the order provided. Each preprocessor should + accept the args (source, file) and should return a tuple (source, + file). + + The argument value should be a list, but a single non-list value + is acceptable and will automatically be converted into a list. + Each item in the list will be passed through + Template._normalizePreprocessor(). The items should either match + one of the following forms: + + - an object with a .preprocess(source, file) method + - a callable with the following signature: + source, file = f(source, file) + + or one of the forms below: + + - a single string denoting the 2 'tokens' for the preprocess + syntax. The tokens should be in the order (placeholderToken, + directiveToken) and should separated with a space: + e.g. '@ %' + klass = Template.compile(src, preprocessors='@ %') + # or + klass = Template.compile(src, preprocessors=['@ %']) + + - a dict with the following keys or an object with the + following attributes (all are optional, but nothing will + happen if you don't provide at least one): + - tokens: same as the single string described above. You can + also provide a tuple of 2 strings. + - searchList: the searchList used for preprocess $placeholders + - compilerSettings: used in the compilation of the intermediate + template + - templateAPIClass: an optional subclass of `Template` + - outputTransformer: a simple hook for passing in a callable + which can do further transformations of the preprocessor + output, or do something else like debug logging. The + default is str(). + + any keyword arguments to Template.compile which you want to + provide for the compilation of the intermediate template. + + klass = Template.compile(src, + preprocessors=[ dict(tokens='@ %', searchList=[...]) ] ) + + """ + errmsg = "arg '%s' must be %s" + + if not isinstance(source, (types.NoneType, basestring)): + raise TypeError(errmsg % ('source', 'string or None')) + + if not isinstance(file, (types.NoneType, basestring, filetype)): + raise TypeError(errmsg % + ('file', 'string, file-like object, or None')) + + if baseclass is Unspecified: + baseclass = klass._CHEETAH_defaultBaseclassForTemplates + if isinstance(baseclass, Template): + baseclass = baseclass.__class__ + + if not isinstance(baseclass, (types.NoneType, basestring, type)): + raise TypeError(errmsg % ('baseclass', 'string, class or None')) + + if cacheCompilationResults is Unspecified: + cacheCompilationResults = klass._CHEETAH_cacheCompilationResults + + if not isinstance(cacheCompilationResults, (int, bool)): + raise TypeError(errmsg % ('cacheCompilationResults', 'boolean')) + + if useCache is Unspecified: + useCache = klass._CHEETAH_useCompilationCache + + if not isinstance(useCache, (int, bool)): + raise TypeError(errmsg % ('useCache', 'boolean')) + + if compilerSettings is Unspecified: + compilerSettings = klass._getCompilerSettings(source, file) or {} + if not isinstance(compilerSettings, dict): + raise TypeError(errmsg % ('compilerSettings', 'dictionary')) + + if compilerClass is Unspecified: + compilerClass = klass._getCompilerClass(source, file) + if preprocessors is Unspecified: + preprocessors = klass._CHEETAH_preprocessors + + if keepRefToGeneratedCode is Unspecified: + keepRefToGeneratedCode = klass._CHEETAH_keepRefToGeneratedCode + + if not isinstance(keepRefToGeneratedCode, (int, bool)): + raise TypeError(errmsg % ('keepReftoGeneratedCode', 'boolean')) + + if not isinstance(moduleName, (types.NoneType, basestring)): + raise TypeError(errmsg % ('moduleName', 'string or None')) + __orig_file__ = None + if not moduleName: + if file and isinstance(file, basestring): + moduleName = convertTmplPathToModuleName(file) + __orig_file__ = file + else: + moduleName = klass._CHEETAH_defaultModuleNameForTemplates + + if className is Unspecified: + className = klass._CHEETAH_defaultClassNameForTemplates + + if not isinstance(className, (types.NoneType, basestring)): + raise TypeError(errmsg % ('className', 'string or None')) + className = re.sub(r'^_+','', className or moduleName) + + if mainMethodName is Unspecified: + mainMethodName = klass._CHEETAH_defaultMainMethodNameForTemplates + + if not isinstance(mainMethodName, (types.NoneType, basestring)): + raise TypeError(errmsg % ('mainMethodName', 'string or None')) + + if moduleGlobals is Unspecified: + moduleGlobals = klass._CHEETAH_defaultModuleGlobalsForTemplates + + if cacheModuleFilesForTracebacks is Unspecified: + cacheModuleFilesForTracebacks = klass._CHEETAH_cacheModuleFilesForTracebacks + + if not isinstance(cacheModuleFilesForTracebacks, (int, bool)): + raise TypeError(errmsg % + ('cacheModuleFilesForTracebacks', 'boolean')) + + if cacheDirForModuleFiles is Unspecified: + cacheDirForModuleFiles = klass._CHEETAH_cacheDirForModuleFiles + + if not isinstance(cacheDirForModuleFiles, (types.NoneType, basestring)): + raise TypeError(errmsg % + ('cacheDirForModuleFiles', 'string or None')) + + ################################################## + ## handle any preprocessors + if preprocessors: + origSrc = source + source, file = klass._preprocessSource(source, file, preprocessors) + + ################################################## + ## compilation, using cache if requested/possible + baseclassValue = None + baseclassName = None + if baseclass: + if isinstance(baseclass, basestring): + baseclassName = baseclass + elif isinstance(baseclass, type): + # @@TR: should soft-code this + baseclassName = 'CHEETAH_dynamicallyAssignedBaseClass_'+baseclass.__name__ + baseclassValue = baseclass + + + cacheHash = None + cacheItem = None + if source or isinstance(file, basestring): + compilerSettingsHash = None + if compilerSettings: + compilerSettingsHash = hashDict(compilerSettings) + + moduleGlobalsHash = None + if moduleGlobals: + moduleGlobalsHash = hashDict(moduleGlobals) + + fileHash = None + if file: + fileHash = str(hash(file)) + if globals()['__checkFileMtime']: + fileHash += str(os.path.getmtime(file)) + + try: + # @@TR: find some way to create a cacheHash that is consistent + # between process restarts. It would allow for caching the + # compiled module on disk and thereby reduce the startup time + # for applications that use a lot of dynamically compiled + # templates. + cacheHash = ''.join([str(v) for v in + [hash(source), + fileHash, + className, + moduleName, + mainMethodName, + hash(compilerClass), + hash(baseclass), + compilerSettingsHash, + moduleGlobalsHash, + hash(cacheDirForModuleFiles), + ]]) + except: + #@@TR: should add some logging to this + pass + outputEncoding = 'ascii' + compiler = None + if useCache and cacheHash and cacheHash in klass._CHEETAH_compileCache: + cacheItem = klass._CHEETAH_compileCache[cacheHash] + generatedModuleCode = cacheItem.code + else: + compiler = compilerClass(source, file, + moduleName=moduleName, + mainClassName=className, + baseclassName=baseclassName, + mainMethodName=mainMethodName, + settings=(compilerSettings or {})) + if commandlineopts: + compiler.setShBang(commandlineopts.shbang) + compiler.compile() + generatedModuleCode = compiler.getModuleCode() + outputEncoding = compiler.getModuleEncoding() + + if not returnAClass: + # This is a bit of a hackish solution to make sure we're setting the proper + # encoding on generated code that is destined to be written to a file + if not outputEncoding == 'ascii': + generatedModuleCode = generatedModuleCode.split('\n') + generatedModuleCode.insert(1, '# -*- coding: %s -*-' % outputEncoding) + generatedModuleCode = '\n'.join(generatedModuleCode) + return generatedModuleCode.encode(outputEncoding) + else: + if cacheItem: + cacheItem.lastCheckoutTime = time.time() + return cacheItem.klass + + try: + klass._CHEETAH_compileLock.acquire() + uniqueModuleName = _genUniqueModuleName(moduleName) + __file__ = uniqueModuleName+'.py' # relative file path with no dir part + + if cacheModuleFilesForTracebacks: + if not os.path.exists(cacheDirForModuleFiles): + raise Exception('%s does not exist'%cacheDirForModuleFiles) + + __file__ = os.path.join(cacheDirForModuleFiles, __file__) + # @@TR: might want to assert that it doesn't already exist + open(__file__, 'w').write(generatedModuleCode) + # @@TR: should probably restrict the perms, etc. + + mod = types.ModuleType(str(uniqueModuleName)) + if moduleGlobals: + for k, v in moduleGlobals.items(): + setattr(mod, k, v) + mod.__file__ = __file__ + if __orig_file__ and os.path.exists(__orig_file__): + # this is used in the WebKit filemonitoring code + mod.__orig_file__ = __orig_file__ + + if baseclass and baseclassValue: + setattr(mod, baseclassName, baseclassValue) + ## + try: + co = compile(generatedModuleCode, __file__, 'exec') + exec(co, mod.__dict__) + except SyntaxError, e: + try: + parseError = genParserErrorFromPythonException( + source, file, generatedModuleCode, exception=e) + except: + updateLinecache(__file__, generatedModuleCode) + e.generatedModuleCode = generatedModuleCode + raise e + else: + raise parseError + except Exception, e: + updateLinecache(__file__, generatedModuleCode) + e.generatedModuleCode = generatedModuleCode + raise + ## + sys.modules[uniqueModuleName] = mod + finally: + klass._CHEETAH_compileLock.release() + + templateClass = getattr(mod, className) + + if (cacheCompilationResults + and cacheHash + and cacheHash not in klass._CHEETAH_compileCache): + + cacheItem = CompileCacheItem() + cacheItem.cacheTime = cacheItem.lastCheckoutTime = time.time() + cacheItem.code = generatedModuleCode + cacheItem.klass = templateClass + templateClass._CHEETAH_isInCompilationCache = True + klass._CHEETAH_compileCache[cacheHash] = cacheItem + else: + templateClass._CHEETAH_isInCompilationCache = False + + if keepRefToGeneratedCode or cacheCompilationResults: + templateClass._CHEETAH_generatedModuleCode = generatedModuleCode + + # If we have a compiler object, let's set it to the compiler class + # to help the directive analyzer code + if compiler: + templateClass._CHEETAH_compilerInstance = compiler + return templateClass + + @classmethod + def subclass(klass, *args, **kws): + """Takes the same args as the .compile() classmethod and returns a + template that is a subclass of the template this method is called from. + + T1 = Template.compile(' foo - $meth1 - bar\n#def meth1: this is T1.meth1') + T2 = T1.subclass('#implements meth1\n this is T2.meth1') + """ + kws['baseclass'] = klass + if isinstance(klass, Template): + templateAPIClass = klass + else: + templateAPIClass = Template + return templateAPIClass.compile(*args, **kws) + + @classmethod + def _preprocessSource(klass, source, file, preprocessors): + """Iterates through the .compile() classmethod's preprocessors argument + and pipes the source code through each each preprocessor. + + It returns the tuple (source, file) which is then used by + Template.compile to finish the compilation. + """ + if not isinstance(preprocessors, (list, tuple)): + preprocessors = [preprocessors] + for preprocessor in preprocessors: + preprocessor = klass._normalizePreprocessorArg(preprocessor) + source, file = preprocessor.preprocess(source, file) + return source, file + + @classmethod + def _normalizePreprocessorArg(klass, arg): + """Used to convert the items in the .compile() classmethod's + preprocessors argument into real source preprocessors. This permits the + use of several shortcut forms for defining preprocessors. + """ + + if hasattr(arg, 'preprocess'): + return arg + elif hasattr(arg, '__call__'): + class WrapperPreprocessor: + def preprocess(self, source, file): + return arg(source, file) + return WrapperPreprocessor() + else: + class Settings(object): + placeholderToken = None + directiveToken = None + settings = Settings() + if isinstance(arg, str) or isinstance(arg, (list, tuple)): + settings.tokens = arg + elif isinstance(arg, dict): + for k, v in arg.items(): + setattr(settings, k, v) + else: + settings = arg + + settings = klass._normalizePreprocessorSettings(settings) + return klass._CHEETAH_defaultPreprocessorClass(settings) + + + @classmethod + def _normalizePreprocessorSettings(klass, settings): + settings.keepRefToGeneratedCode = True + + def normalizeSearchList(searchList): + if not isinstance(searchList, (list, tuple)): + searchList = [searchList] + return searchList + + def normalizeTokens(tokens): + if isinstance(tokens, str): + return tokens.split() # space delimited string e.g.'@ %' + elif isinstance(tokens, (list, tuple)): + return tokens + else: + raise PreprocessError('invalid tokens argument: %r'%tokens) + + if hasattr(settings, 'tokens'): + (settings.placeholderToken, + settings.directiveToken) = normalizeTokens(settings.tokens) + + if (not getattr(settings, 'compilerSettings', None) + and not getattr(settings, 'placeholderToken', None) ): + + raise TypeError( + 'Preprocessor requires either a "tokens" or a "compilerSettings" arg.' + ' Neither was provided.') + + if not hasattr(settings, 'templateInitArgs'): + settings.templateInitArgs = {} + if 'searchList' not in settings.templateInitArgs: + if not hasattr(settings, 'searchList') and hasattr(settings, 'namespaces'): + settings.searchList = settings.namespaces + elif not hasattr(settings, 'searchList'): + settings.searchList = [] + settings.templateInitArgs['searchList'] = settings.searchList + settings.templateInitArgs['searchList'] = ( + normalizeSearchList(settings.templateInitArgs['searchList'])) + + if not hasattr(settings, 'outputTransformer'): + settings.outputTransformer = unicode + + if not hasattr(settings, 'templateAPIClass'): + class PreprocessTemplateAPIClass(klass): pass + settings.templateAPIClass = PreprocessTemplateAPIClass + + if not hasattr(settings, 'compilerSettings'): + settings.compilerSettings = {} + + klass._updateSettingsWithPreprocessTokens( + compilerSettings=settings.compilerSettings, + placeholderToken=settings.placeholderToken, + directiveToken=settings.directiveToken + ) + return settings + + @classmethod + def _updateSettingsWithPreprocessTokens( + klass, compilerSettings, placeholderToken, directiveToken): + + if (placeholderToken and 'cheetahVarStartToken' not in compilerSettings): + compilerSettings['cheetahVarStartToken'] = placeholderToken + if directiveToken: + if 'directiveStartToken' not in compilerSettings: + compilerSettings['directiveStartToken'] = directiveToken + if 'directiveEndToken' not in compilerSettings: + compilerSettings['directiveEndToken'] = directiveToken + if 'commentStartToken' not in compilerSettings: + compilerSettings['commentStartToken'] = directiveToken*2 + if 'multiLineCommentStartToken' not in compilerSettings: + compilerSettings['multiLineCommentStartToken'] = ( + directiveToken+'*') + if 'multiLineCommentEndToken' not in compilerSettings: + compilerSettings['multiLineCommentEndToken'] = ( + '*'+directiveToken) + if 'EOLSlurpToken' not in compilerSettings: + compilerSettings['EOLSlurpToken'] = directiveToken + + @classmethod + def _addCheetahPlumbingCodeToClass(klass, concreteTemplateClass): + """If concreteTemplateClass is not a subclass of Cheetah.Template, add + the required cheetah methods and attributes to it. + + This is called on each new template class after it has been compiled. + If concreteTemplateClass is not a subclass of Cheetah.Template but + already has method with the same name as one of the required cheetah + methods, this will skip that method. + """ + for methodname in klass._CHEETAH_requiredCheetahMethods: + if not hasattr(concreteTemplateClass, methodname): + method = getattr(Template, methodname) + newMethod = createMethod(method.im_func, concreteTemplateClass) + setattr(concreteTemplateClass, methodname, newMethod) + + for classMethName in klass._CHEETAH_requiredCheetahClassMethods: + if not hasattr(concreteTemplateClass, classMethName): + meth = getattr(klass, classMethName) + setattr(concreteTemplateClass, classMethName, classmethod(meth.im_func)) + + for attrname in klass._CHEETAH_requiredCheetahClassAttributes: + attrname = '_CHEETAH_'+attrname + if not hasattr(concreteTemplateClass, attrname): + attrVal = getattr(klass, attrname) + setattr(concreteTemplateClass, attrname, attrVal) + + if (not hasattr(concreteTemplateClass, '__str__') + or concreteTemplateClass.__str__ is object.__str__): + + mainMethNameAttr = '_mainCheetahMethod_for_'+concreteTemplateClass.__name__ + mainMethName = getattr(concreteTemplateClass, mainMethNameAttr, None) + if mainMethName: + def __str__(self): + rc = getattr(self, mainMethName)() + if isinstance(rc, unicode): + return rc.encode('utf-8') + return rc + def __unicode__(self): + return getattr(self, mainMethName)() + elif (hasattr(concreteTemplateClass, 'respond') + and concreteTemplateClass.respond!=Servlet.respond): + def __str__(self): + rc = self.respond() + if isinstance(rc, unicode): + return rc.encode('utf-8') + return rc + def __unicode__(self): + return self.respond() + else: + def __str__(self): + rc = None + if hasattr(self, mainMethNameAttr): + rc = getattr(self, mainMethNameAttr)() + elif hasattr(self, 'respond'): + rc = self.respond() + else: + rc = super(self.__class__, self).__str__() + if isinstance(rc, unicode): + return rc.encode('utf-8') + return rc + def __unicode__(self): + if hasattr(self, mainMethNameAttr): + return getattr(self, mainMethNameAttr)() + elif hasattr(self, 'respond'): + return self.respond() + else: + return super(self.__class__, self).__unicode__() + + __str__ = createMethod(__str__, concreteTemplateClass) + __unicode__ = createMethod(__unicode__, concreteTemplateClass) + setattr(concreteTemplateClass, '__str__', __str__) + setattr(concreteTemplateClass, '__unicode__', __unicode__) + + + def __init__(self, source=None, + + namespaces=None, searchList=None, + # use either or. They are aliases for the same thing. + + file=None, + filter='RawOrEncodedUnicode', # which filter from Cheetah.Filters + filtersLib=Filters, + errorCatcher=None, + + compilerSettings=Unspecified, # control the behaviour of the compiler + _globalSetVars=None, # used internally for #include'd templates + _preBuiltSearchList=None # used internally for #include'd templates + ): + """a) compiles a new template OR b) instantiates an existing template. + + Read this docstring carefully as there are two distinct usage patterns. + You should also read this class' main docstring. + + a) to compile a new template: + t = Template(source=aSourceString) + # or + t = Template(file='some/path') + # or + t = Template(file=someFileObject) + # or + namespaces = [{'foo':'bar'}] + t = Template(source=aSourceString, namespaces=namespaces) + # or + t = Template(file='some/path', namespaces=namespaces) + + print t + + b) to create an instance of an existing, precompiled template class: + ## i) first you need a reference to a compiled template class: + tclass = Template.compile(source=src) # or just Template.compile(src) + # or + tclass = Template.compile(file='some/path') + # or + tclass = Template.compile(file=someFileObject) + # or + # if you used the command line compiler or have Cheetah's ImportHooks + # installed your template class is also available via Python's + # standard import mechanism: + from ACompileTemplate import AcompiledTemplate as tclass + + ## ii) then you create an instance + t = tclass(namespaces=namespaces) + # or + t = tclass(namespaces=namespaces, filter='RawOrEncodedUnicode') + print t + + Arguments: + for usage pattern a) + If you are compiling a new template, you must provide either a + 'source' or 'file' arg, but not both: + - source (string or None) + - file (string path, file-like object, or None) + + Optional args (see below for more) : + - compilerSettings + Default: Template._CHEETAH_compilerSettings=None + + a dictionary of settings to override those defined in + DEFAULT_COMPILER_SETTINGS. See + Cheetah.Template.DEFAULT_COMPILER_SETTINGS and the Users' Guide + for details. + + You can pass the source arg in as a positional arg with this usage + pattern. Use keywords for all other args. + + for usage pattern b) + Do not use positional args with this usage pattern, unless your + template subclasses something other than Cheetah.Template and you + want to pass positional args to that baseclass. E.g.: + dictTemplate = Template.compile('hello $name from $caller', baseclass=dict) + tmplvars = dict(name='world', caller='me') + print dictTemplate(tmplvars) + This usage requires all Cheetah args to be passed in as keyword args. + + optional args for both usage patterns: + + - namespaces (aka 'searchList') + Default: None + + an optional list of namespaces (dictionaries, objects, modules, + etc.) which Cheetah will search through to find the variables + referenced in $placeholders. + + If you provide a single namespace instead of a list, Cheetah will + automatically convert it into a list. + + NOTE: Cheetah does NOT force you to use the namespaces search list + and related features. It's on by default, but you can turn if off + using the compiler settings useSearchList=False or + useNameMapper=False. + + - filter + Default: 'EncodeUnicode' + + Which filter should be used for output filtering. This should + either be a string which is the name of a filter in the + 'filtersLib' or a subclass of Cheetah.Filters.Filter. . See the + Users' Guide for more details. + + - filtersLib + Default: Cheetah.Filters + + A module containing subclasses of Cheetah.Filters.Filter. See the + Users' Guide for more details. + + - errorCatcher + Default: None + + This is a debugging tool. See the Users' Guide for more details. + Do not use this or the #errorCatcher diretive with live + production systems. + + Do NOT mess with the args _globalSetVars or _preBuiltSearchList! + + + """ + errmsg = "arg '%s' must be %s" + errmsgextra = errmsg + "\n%s" + + if not isinstance(source, (types.NoneType, basestring)): + raise TypeError(errmsg % ('source', 'string or None')) + + if not isinstance(source, (types.NoneType, basestring, filetype)): + raise TypeError(errmsg % + ('file', 'string, file open for reading, or None')) + + if not isinstance(filter, (basestring, types.TypeType)) and not \ + (isinstance(filter, type) and issubclass(filter, Filters.Filter)): + raise TypeError(errmsgextra % + ('filter', 'string or class', + '(if class, must be subclass of Cheetah.Filters.Filter)')) + if not isinstance(filtersLib, (basestring, types.ModuleType)): + raise TypeError(errmsgextra % + ('filtersLib', 'string or module', + '(if module, must contain subclasses of Cheetah.Filters.Filter)')) + + if not errorCatcher is None: + err = True + if isinstance(errorCatcher, (basestring, types.TypeType)): + err = False + if isinstance(errorCatcher, type) and \ + issubclass(errorCatcher, ErrorCatchers.ErrorCatcher): + err = False + if err: + raise TypeError(errmsgextra % + ('errorCatcher', 'string, class or None', + '(if class, must be subclass of Cheetah.ErrorCatchers.ErrorCatcher)')) + if compilerSettings is not Unspecified: + if not isinstance(compilerSettings, types.DictType): + raise TypeError(errmsg % + ('compilerSettings', 'dictionary')) + + if source is not None and file is not None: + raise TypeError("you must supply either a source string or the" + + " 'file' keyword argument, but not both") + + ################################################## + ## Do superclass initialization. + super(Template, self).__init__() + + ################################################## + ## Do required version check + if not hasattr(self, '_CHEETAH_versionTuple'): + try: + mod = sys.modules[self.__class__.__module__] + compiledVersion = mod.__CHEETAH_version__ + compiledVersionTuple = convertVersionStringToTuple(compiledVersion) + if compiledVersionTuple < MinCompatibleVersionTuple: + raise AssertionError( + 'This template was compiled with Cheetah version' + ' %s. Templates compiled before version %s must be recompiled.'%( + compiledVersion, MinCompatibleVersion)) + except AssertionError: + raise + except: + pass + + ################################################## + ## Setup instance state attributes used during the life of template + ## post-compile + if searchList: + for namespace in searchList: + if isinstance(namespace, dict): + intersection = self.Reserved_SearchList & set(namespace.keys()) + warn = False + if intersection: + warn = True + if isinstance(compilerSettings, dict) and compilerSettings.get('prioritizeSearchListOverSelf'): + warn = False + if warn: + logging.info(''' The following keys are members of the Template class and will result in NameMapper collisions! ''') + logging.info(''' > %s ''' % ', '.join(list(intersection))) + logging.info(''' Please change the key's name or use the compiler setting "prioritizeSearchListOverSelf=True" to prevent the NameMapper from using ''') + logging.info(''' the Template member in place of your searchList variable ''') + + + self._initCheetahInstance( + searchList=searchList, namespaces=namespaces, + filter=filter, filtersLib=filtersLib, + errorCatcher=errorCatcher, + _globalSetVars=_globalSetVars, + compilerSettings=compilerSettings, + _preBuiltSearchList=_preBuiltSearchList) + + ################################################## + ## Now, compile if we're meant to + if (source is not None) or (file is not None): + self._compile(source, file, compilerSettings=compilerSettings) + + def generatedModuleCode(self): + """Return the module code the compiler generated, or None if no + compilation took place. + """ + + return self._CHEETAH_generatedModuleCode + + def generatedClassCode(self): + """Return the class code the compiler generated, or None if no + compilation took place. + """ + + return self._CHEETAH_generatedModuleCode[ + self._CHEETAH_generatedModuleCode.find('\nclass '): + self._CHEETAH_generatedModuleCode.find('\n## END CLASS DEFINITION')] + + def searchList(self): + """Return a reference to the searchlist + """ + return self._CHEETAH__searchList + + def errorCatcher(self): + """Return a reference to the current errorCatcher + """ + return self._CHEETAH__errorCatcher + + ## cache methods ## + def _getCacheStore(self): + if not self._CHEETAH__cacheStore: + if self._CHEETAH_cacheStore is not None: + self._CHEETAH__cacheStore = self._CHEETAH_cacheStore + else: + # @@TR: might want to provide a way to provide init args + self._CHEETAH__cacheStore = self._CHEETAH_cacheStoreClass() + + return self._CHEETAH__cacheStore + + def _getCacheStoreIdPrefix(self): + if self._CHEETAH_cacheStoreIdPrefix is not None: + return self._CHEETAH_cacheStoreIdPrefix + else: + return str(id(self)) + + def _createCacheRegion(self, regionID): + return self._CHEETAH_cacheRegionClass( + regionID=regionID, + templateCacheIdPrefix=self._getCacheStoreIdPrefix(), + cacheStore=self._getCacheStore()) + + def getCacheRegion(self, regionID, cacheInfo=None, create=True): + cacheRegion = self._CHEETAH__cacheRegions.get(regionID) + if not cacheRegion and create: + cacheRegion = self._createCacheRegion(regionID) + self._CHEETAH__cacheRegions[regionID] = cacheRegion + return cacheRegion + + def getCacheRegions(self): + """Returns a dictionary of the 'cache regions' initialized in a + template. + + Each #cache directive block or $*cachedPlaceholder is a separate 'cache + region'. + """ + # returns a copy to prevent users mucking it up + return self._CHEETAH__cacheRegions.copy() + + def refreshCache(self, cacheRegionId=None, cacheItemId=None): + """Refresh a cache region or a specific cache item within a region. + """ + + if not cacheRegionId: + for cacheRegion in self.getCacheRegions().itervalues(): + cacheRegion.clear() + else: + cregion = self._CHEETAH__cacheRegions.get(cacheRegionId) + if not cregion: + return + if not cacheItemId: # clear the desired region and all its cacheItems + cregion.clear() + else: # clear one specific cache of a specific region + cache = cregion.getCacheItem(cacheItemId) + if cache: + cache.clear() + + ## end cache methods ## + + def shutdown(self): + """Break reference cycles before discarding a servlet. + """ + try: + Servlet.shutdown(self) + except: + pass + self._CHEETAH__searchList = None + self.__dict__ = {} + + ## utility functions ## + + def getVar(self, varName, default=Unspecified, autoCall=True): + """Get a variable from the searchList. If the variable can't be found + in the searchList, it returns the default value if one was given, or + raises NameMapper.NotFound. + """ + + try: + return valueFromSearchList(self.searchList(), varName.replace('$', ''), autoCall) + except NotFound: + if default is not Unspecified: + return default + else: + raise + + def varExists(self, varName, autoCall=True): + """Test if a variable name exists in the searchList. + """ + try: + valueFromSearchList(self.searchList(), varName.replace('$', ''), autoCall) + return True + except NotFound: + return False + + + hasVar = varExists + + + def i18n(self, message, + plural=None, + n=None, + + id=None, + domain=None, + source=None, + target=None, + comment=None + ): + """This is just a stub at this time. + + plural = the plural form of the message + n = a sized argument to distinguish between single and plural forms + + id = msgid in the translation catalog + domain = translation domain + source = source lang + target = a specific target lang + comment = a comment to the translation team + + See the following for some ideas + http://www.zope.org/DevHome/Wikis/DevSite/Projects/ComponentArchitecture/ZPTInternationalizationSupport + + Other notes: + - There is no need to replicate the i18n:name attribute from plone / PTL, + as cheetah placeholders serve the same purpose + + + """ + + return message + + def getFileContents(self, path): + """A hook for getting the contents of a file. The default + implementation just uses the Python open() function to load local files. + This method could be reimplemented to allow reading of remote files via + various protocols, as PHP allows with its 'URL fopen wrapper' + """ + + fp = open(path, 'r') + output = fp.read() + fp.close() + return output + + def runAsMainProgram(self): + """Allows the Template to function as a standalone command-line program + for static page generation. + + Type 'python yourtemplate.py --help to see what it's capabable of. + """ + + from TemplateCmdLineIface import CmdLineIface + CmdLineIface(templateObj=self).run() + + ################################################## + ## internal methods -- not to be called by end-users + + def _initCheetahInstance(self, + searchList=None, + namespaces=None, + filter='RawOrEncodedUnicode', # which filter from Cheetah.Filters + filtersLib=Filters, + errorCatcher=None, + _globalSetVars=None, + compilerSettings=None, + _preBuiltSearchList=None): + """Sets up the instance attributes that cheetah templates use at + run-time. + + This is automatically called by the __init__ method of compiled + templates. + + Note that the names of instance attributes used by Cheetah are prefixed + with '_CHEETAH__' (2 underscores), where class attributes are prefixed + with '_CHEETAH_' (1 underscore). + """ + if getattr(self, '_CHEETAH__instanceInitialized', False): + return + + if namespaces is not None: + assert searchList is None, ( + 'Provide "namespaces" or "searchList", not both!') + searchList = namespaces + if searchList is not None and not isinstance(searchList, (list, tuple)): + searchList = [searchList] + + self._CHEETAH__globalSetVars = {} + if _globalSetVars is not None: + # this is intended to be used internally by Nested Templates in #include's + self._CHEETAH__globalSetVars = _globalSetVars + + if _preBuiltSearchList is not None: + # happens with nested Template obj creation from #include's + self._CHEETAH__searchList = list(_preBuiltSearchList) + self._CHEETAH__searchList.append(self) + else: + # create our own searchList + self._CHEETAH__searchList = [self._CHEETAH__globalSetVars, self] + if searchList is not None: + if isinstance(compilerSettings, dict) and compilerSettings.get('prioritizeSearchListOverSelf'): + self._CHEETAH__searchList = searchList + self._CHEETAH__searchList + else: + self._CHEETAH__searchList.extend(list(searchList)) + self._CHEETAH__cheetahIncludes = {} + self._CHEETAH__cacheRegions = {} + self._CHEETAH__indenter = Indenter() + + # @@TR: consider allowing simple callables as the filter argument + self._CHEETAH__filtersLib = filtersLib + self._CHEETAH__filters = {} + if isinstance(filter, basestring): + filterName = filter + klass = getattr(self._CHEETAH__filtersLib, filterName) + else: + klass = filter + filterName = klass.__name__ + self._CHEETAH__currentFilter = self._CHEETAH__filters[filterName] = klass(self).filter + self._CHEETAH__initialFilter = self._CHEETAH__currentFilter + + self._CHEETAH__errorCatchers = {} + if errorCatcher: + if isinstance(errorCatcher, basestring): + errorCatcherClass = getattr(ErrorCatchers, errorCatcher) + elif isinstance(errorCatcher, type): + errorCatcherClass = errorCatcher + + self._CHEETAH__errorCatcher = ec = errorCatcherClass(self) + self._CHEETAH__errorCatchers[errorCatcher.__class__.__name__] = ec + + else: + self._CHEETAH__errorCatcher = None + self._CHEETAH__initErrorCatcher = self._CHEETAH__errorCatcher + + if not hasattr(self, 'transaction'): + self.transaction = None + self._CHEETAH__instanceInitialized = True + self._CHEETAH__isBuffering = False + self._CHEETAH__isControlledByWebKit = False + + self._CHEETAH__cacheStore = None + if self._CHEETAH_cacheStore is not None: + self._CHEETAH__cacheStore = self._CHEETAH_cacheStore + + def _compile(self, source=None, file=None, compilerSettings=Unspecified, + moduleName=None, mainMethodName=None): + """Compile the template. This method is automatically called by + Template.__init__ it is provided with 'file' or 'source' args. + + USERS SHOULD *NEVER* CALL THIS METHOD THEMSELVES. Use Template.compile + instead. + """ + if compilerSettings is Unspecified: + compilerSettings = self._getCompilerSettings(source, file) or {} + mainMethodName = mainMethodName or self._CHEETAH_defaultMainMethodName + self._fileMtime = None + self._fileDirName = None + self._fileBaseName = None + if file and isinstance(file, basestring): + file = self.serverSidePath(file) + self._fileMtime = os.path.getmtime(file) + self._fileDirName, self._fileBaseName = os.path.split(file) + self._filePath = file + templateClass = self.compile(source, file, + moduleName=moduleName, + mainMethodName=mainMethodName, + compilerSettings=compilerSettings, + keepRefToGeneratedCode=True) + + if not self.__class__ == Template: + # Only propogate attributes if we're in a subclass of + # Template + for k, v in self.__class__.__dict__.iteritems(): + if not v or k.startswith('__'): + continue + ## Propogate the class attributes to the instance + ## since we're about to obliterate self.__class__ + ## (see: cheetah.Tests.Tepmlate.SubclassSearchListTest) + setattr(self, k, v) + + self.__class__ = templateClass + # must initialize it so instance attributes are accessible + templateClass.__init__(self, + #_globalSetVars=self._CHEETAH__globalSetVars, + #_preBuiltSearchList=self._CHEETAH__searchList + ) + if not hasattr(self, 'transaction'): + self.transaction = None + + def _handleCheetahInclude(self, srcArg, trans=None, includeFrom='file', raw=False): + """Called at runtime to handle #include directives. + """ + _includeID = srcArg + if _includeID not in self._CHEETAH__cheetahIncludes: + if not raw: + if includeFrom == 'file': + source = None + if isinstance(srcArg, basestring): + if hasattr(self, 'serverSidePath'): + file = path = self.serverSidePath(srcArg) + else: + file = path = os.path.normpath(srcArg) + else: + file = srcArg ## a file-like object + else: + source = srcArg + file = None + # @@TR: might want to provide some syntax for specifying the + # Template class to be used for compilation so compilerSettings + # can be changed. + compiler = self._getTemplateAPIClassForIncludeDirectiveCompilation(source, file) + nestedTemplateClass = compiler.compile(source=source, file=file) + nestedTemplate = nestedTemplateClass(_preBuiltSearchList=self.searchList(), + _globalSetVars=self._CHEETAH__globalSetVars) + # Set the inner template filters to the initial filter of the + # outer template: + # this is the only really safe way to use + # filter='WebSafe'. + nestedTemplate._CHEETAH__initialFilter = self._CHEETAH__initialFilter + nestedTemplate._CHEETAH__currentFilter = self._CHEETAH__initialFilter + self._CHEETAH__cheetahIncludes[_includeID] = nestedTemplate + else: + if includeFrom == 'file': + path = self.serverSidePath(srcArg) + self._CHEETAH__cheetahIncludes[_includeID] = self.getFileContents(path) + else: + self._CHEETAH__cheetahIncludes[_includeID] = srcArg + ## + if not raw: + self._CHEETAH__cheetahIncludes[_includeID].respond(trans) + else: + trans.response().write(self._CHEETAH__cheetahIncludes[_includeID]) + + def _getTemplateAPIClassForIncludeDirectiveCompilation(self, source, file): + """Returns the subclass of Template which should be used to compile + #include directives. + + This abstraction allows different compiler settings to be used in the + included template than were used in the parent. + """ + if issubclass(self.__class__, Template): + return self.__class__ + else: + return Template + + ## functions for using templates as CGI scripts + def webInput(self, names, namesMulti=(), default='', src='f', + defaultInt=0, defaultFloat=0.00, badInt=0, badFloat=0.00, debug=False): + """Method for importing web transaction variables in bulk. + + This works for GET/POST fields both in Webware servlets and in CGI + scripts, and for cookies and session variables in Webware servlets. If + you try to read a cookie or session variable in a CGI script, you'll get + a RuntimeError. 'In a CGI script' here means 'not running as a Webware + servlet'. If the CGI environment is not properly set up, Cheetah will + act like there's no input. + + The public method provided is: + + def webInput(self, names, namesMulti=(), default='', src='f', + defaultInt=0, defaultFloat=0.00, badInt=0, badFloat=0.00, debug=False): + + This method places the specified GET/POST fields, cookies or session + variables into a dictionary, which is both returned and put at the + beginning of the searchList. It handles: + + * single vs multiple values + * conversion to integer or float for specified names + * default values/exceptions for missing or bad values + * printing a snapshot of all values retrieved for debugging + + All the 'default*' and 'bad*' arguments have 'use or raise' behavior, + meaning that if they're a subclass of Exception, they're raised. If + they're anything else, that value is substituted for the missing/bad + value. + + + The simplest usage is: + + #silent $webInput(['choice']) + $choice + + dic = self.webInput(['choice']) + write(dic['choice']) + + Both these examples retrieves the GET/POST field 'choice' and print it. + If you leave off the'#silent', all the values would be printed too. But + a better way to preview the values is + + #silent $webInput(['name'], $debug=1) + + because this pretty-prints all the values inside HTML 
     tags.
+
+        ** KLUDGE: 'debug' is supposed to insert into the template output, but it
+        wasn't working so I changed it to a'print' statement.  So the debugging
+        output will appear wherever standard output is pointed, whether at the
+        terminal, in a Webware log file, or whatever. ***
+
+        Since we didn't specify any coversions, the value is a string.  It's a
+        'single' value because we specified it in 'names' rather than
+        'namesMulti'. Single values work like this:
+        
+            * If one value is found, take it.
+            * If several values are found, choose one arbitrarily and ignore the rest.
+            * If no values are found, use or raise the appropriate 'default*' value.
+
+        Multi values work like this:
+            * If one value is found, put it in a list.
+            * If several values are found, leave them in a list.
+            * If no values are found, use the empty list ([]).  The 'default*' 
+              arguments are *not* consulted in this case.
+
+        Example: assume 'days' came from a set of checkboxes or a multiple combo
+        box on a form, and the user  chose'Monday', 'Tuesday' and 'Thursday'.
+
+            #silent $webInput([], ['days'])
+            The days you chose are: #slurp
+            #for $day in $days
+            $day #slurp
+            #end for
+
+            dic = self.webInput([], ['days'])
+            write('The days you chose are: ')
+            for day in dic['days']:
+                write(day + ' ')
+
+        Both these examples print:  'The days you chose are: Monday Tuesday Thursday'.
+
+        By default, missing strings are replaced by '' and missing/bad numbers
+        by zero.  (A'bad number' means the converter raised an exception for
+        it, usually because of non-numeric characters in the value.)  This
+        mimics Perl/PHP behavior, and simplifies coding for many applications
+        where missing/bad values *should* be blank/zero.  In those relatively
+        few cases where you must distinguish between empty-string/zero on the
+        one hand and missing/bad on the other, change the appropriate
+        'default*' and 'bad*' arguments to something like: 
+
+            * None
+            * another constant value
+            * $NonNumericInputError/self.NonNumericInputError
+            * $ValueError/ValueError
+            
+        (NonNumericInputError is defined in this class and is useful for
+        distinguishing between bad input vs a TypeError/ValueError thrown for
+        some other rason.)
+
+        Here's an example using multiple values to schedule newspaper
+        deliveries.  'checkboxes' comes from a form with checkboxes for all the
+        days of the week.  The days the user previously chose are preselected.
+        The user checks/unchecks boxes as desired and presses Submit.  The value
+        of 'checkboxes' is a list of checkboxes that were checked when Submit
+        was pressed.  Our task now is to turn on the days the user checked, turn
+        off the days he unchecked, and leave on or off the days he didn't
+        change.
+
+            dic = self.webInput([], ['dayCheckboxes'])
+            wantedDays = dic['dayCheckboxes'] # The days the user checked.
+            for day, on in self.getAllValues():
+                if   not on and wantedDays.has_key(day):
+                    self.TurnOn(day)
+                    # ... Set a flag or insert a database record ...
+                elif on and not wantedDays.has_key(day):
+                    self.TurnOff(day)
+                    # ... Unset a flag or delete a database record ...
+
+        'source' allows you to look up the variables from a number of different
+        sources:
+            'f'   fields (CGI GET/POST parameters)
+            'c'   cookies
+            's'   session variables
+            'v'   'values', meaning fields or cookies
+
+        In many forms, you're dealing only with strings, which is why the
+        'default' argument is third and the numeric arguments are banished to
+        the end.  But sometimes you want automatic number conversion, so that
+        you can do numeric comparisions in your templates without having to
+        write a bunch of conversion/exception handling code.  Example:
+
+            #silent $webInput(['name', 'height:int'])
+            $name is $height cm tall.
+            #if $height >= 300
+            Wow, you're tall!
+            #else
+            Pshaw, you're short.
+            #end if
+
+            dic = self.webInput(['name', 'height:int'])
+            name = dic[name]
+            height = dic[height]
+            write('%s is %s cm tall.' % (name, height))
+            if height > 300:
+                write('Wow, you're tall!')
+            else:
+                write('Pshaw, you're short.')
+
+        To convert a value to a number, suffix ':int' or ':float' to the name.
+        The method will search first for a 'height:int' variable and then for a
+        'height' variable.  (It will be called 'height' in the final
+        dictionary.)  If a numeric conversion fails, use or raise 'badInt' or
+        'badFloat'.  Missing values work the same way as for strings, except the
+        default is 'defaultInt' or 'defaultFloat' instead of 'default'.
+
+        If a name represents an uploaded file, the entire file will be read into
+        memory.  For more sophistocated file-upload handling, leave that name
+        out of the list and do your own handling, or wait for
+        Cheetah.Utils.UploadFileMixin.
+
+        This only in a subclass that also inherits from Webware's Servlet or
+        HTTPServlet.  Otherwise you'll get an AttributeError on 'self.request'.
+
+        EXCEPTIONS: ValueError if 'source' is not one of the stated characters.
+        TypeError if a conversion suffix is not ':int' or ':float'.
+
+        FUTURE EXPANSION: a future version of this method may allow source
+        cascading; e.g., 'vs' would look first in 'values' and then in session
+        variables.
+
+        Meta-Data
+        ================================================================================
+        Author: Mike Orr 
+        License: This software is released for unlimited distribution under the
+                 terms of the MIT license.  See the LICENSE file.
+        Version: $Revision: 1.186 $
+        Start Date: 2002/03/17
+        Last Revision Date: $Date: 2008/03/10 04:48:11 $
+        """ 
+        src = src.lower()
+        isCgi = not self._CHEETAH__isControlledByWebKit
+        if   isCgi and src in ('f', 'v'):
+            global _formUsedByWebInput
+            if _formUsedByWebInput is None:
+                _formUsedByWebInput = cgi.FieldStorage()
+            source, func = 'field',   _formUsedByWebInput.getvalue
+        elif isCgi and src == 'c':
+            raise RuntimeError("can't get cookies from a CGI script")
+        elif isCgi and src == 's':
+            raise RuntimeError("can't get session variables from a CGI script")
+        elif isCgi and src == 'v':
+            source, func = 'value',   self.request().value
+        elif isCgi and src == 's':
+            source, func = 'session', self.request().session().value
+        elif src == 'f':
+            source, func = 'field',   self.request().field
+        elif src == 'c':
+            source, func = 'cookie',  self.request().cookie
+        elif src == 'v':
+            source, func = 'value',   self.request().value
+        elif src == 's':
+            source, func = 'session', self.request().session().value
+        else:
+            raise TypeError("arg 'src' invalid")
+        sources = source + 's'
+        converters = {
+            '': _Converter('string', None, default,      default ),
+            'int': _Converter('int',     int, defaultInt,   badInt  ),
+            'float': _Converter('float', float, defaultFloat, badFloat),  }
+        #pprint.pprint(locals());  return {}
+        dic = {} # Destination.
+        for name in names:
+            k, v = _lookup(name, func, False, converters)
+            dic[k] = v
+        for name in namesMulti:
+            k, v = _lookup(name, func, True, converters)
+            dic[k] = v
+        # At this point, 'dic' contains all the keys/values we want to keep.
+        # We could split the method into a superclass
+        # method for Webware/WebwareExperimental and a subclass for Cheetah.
+        # The superclass would merely 'return dic'.  The subclass would
+        # 'dic = super(ThisClass, self).webInput(names, namesMulti, ...)'
+        # and then the code below.
+        if debug:
+           print("
    \n" + pprint.pformat(dic) + "\n
    \n\n")
+        self.searchList().insert(0, dic)
+        return dic
+
+T = Template   # Short and sweet for debugging at the >>> prompt.
+Template.Reserved_SearchList = set(dir(Template))
+
+def genParserErrorFromPythonException(source, file, generatedPyCode, exception):
+
+    #print dir(exception)
+    
+    filename = isinstance(file, (str, unicode)) and file or None
+
+    sio = StringIO.StringIO()
+    traceback.print_exc(1, sio)
+    formatedExc = sio.getvalue()
+    
+    if hasattr(exception, 'lineno'):
+        pyLineno = exception.lineno
+    else:
+        pyLineno = int(re.search('[ \t]*File.*line (\d+)', formatedExc).group(1))
+       
+    lines = generatedPyCode.splitlines()
+    
+    prevLines = []                  # (i, content)
+    for i in range(1, 4):
+        if pyLineno-i <=0:
+            break
+        prevLines.append( (pyLineno+1-i, lines[pyLineno-i]) )
+    
+    nextLines = []                  # (i, content)
+    for i in range(1, 4):
+        if not pyLineno+i < len(lines):
+            break
+        nextLines.append( (pyLineno+i, lines[pyLineno+i]) )
+    nextLines.reverse()
+    report = 'Line|Python Code\n'
+    report += '----|-------------------------------------------------------------\n'
+    while prevLines:
+        lineInfo = prevLines.pop()
+        report += "%(row)-4d|%(line)s\n"% {'row':lineInfo[0], 'line':lineInfo[1]}
+
+    if hasattr(exception, 'offset'):
+        report += ' '*(3+(exception.offset or 0)) + '^\n'
+    
+    while nextLines:
+        lineInfo = nextLines.pop()
+        report += "%(row)-4d|%(line)s\n"% {'row':lineInfo[0], 'line':lineInfo[1]}
+    
+    
+    message = [
+        "Error in the Python code which Cheetah generated for this template:",
+        '='*80,
+        '',
+        str(exception),
+        '',                            
+        report,
+        '='*80,
+        ]
+    cheetahPosMatch = re.search('line (\d+), col (\d+)', formatedExc)
+    if cheetahPosMatch:
+        lineno = int(cheetahPosMatch.group(1))
+        col = int(cheetahPosMatch.group(2))
+        #if hasattr(exception, 'offset'):
+        #    col = exception.offset
+        message.append('\nHere is the corresponding Cheetah code:\n')
+    else:
+        lineno = None
+        col = None
+        cheetahPosMatch = re.search('line (\d+), col (\d+)',
+                                    '\n'.join(lines[max(pyLineno-2, 0):]))
+        if cheetahPosMatch:
+            lineno = int(cheetahPosMatch.group(1))
+            col = int(cheetahPosMatch.group(2))
+            message.append('\nHere is the corresponding Cheetah code.')
+            message.append('** I had to guess the line & column numbers,'
+                           ' so they are probably incorrect:\n')
+
+    
+    message = '\n'.join(message)
+    reader = SourceReader(source, filename=filename)
+    return ParseError(reader, message, lineno=lineno, col=col)
+    
+
+# vim: shiftwidth=4 tabstop=4 expandtab

=== added file 'cheetah/TemplateCmdLineIface.py'
--- cheetah/TemplateCmdLineIface.py	1970-01-01 00:00:00 +0000
+++ cheetah/TemplateCmdLineIface.py	2010-10-19 19:11:10 +0000
@@ -0,0 +1,107 @@
+# $Id: TemplateCmdLineIface.py,v 1.13 2006/01/10 20:34:35 tavis_rudd Exp $
+
+"""Provides a command line interface to compiled Cheetah template modules.
+
+Meta-Data
+================================================================================
+Author: Tavis Rudd 
+Version: $Revision: 1.13 $
+Start Date: 2001/12/06
+Last Revision Date: $Date: 2006/01/10 20:34:35 $
+"""
+__author__ = "Tavis Rudd "
+__revision__ = "$Revision: 1.13 $"[11:-2]
+
+import sys
+import os
+import getopt
+import os.path
+try:
+    from cPickle import load
+except ImportError:
+    from pickle import load
+
+from Cheetah.Version import Version
+
+class Error(Exception):
+    pass
+
+class CmdLineIface:
+    """A command line interface to compiled Cheetah template modules."""
+
+    def __init__(self, templateObj,
+                 scriptName=os.path.basename(sys.argv[0]),
+                 cmdLineArgs=sys.argv[1:]):
+
+        self._template = templateObj
+        self._scriptName = scriptName
+        self._cmdLineArgs = cmdLineArgs
+
+    def run(self):
+        """The main program controller."""
+        
+        self._processCmdLineArgs()
+        print(self._template)
+        
+    def _processCmdLineArgs(self):
+        try:
+            self._opts, self._args = getopt.getopt(
+                self._cmdLineArgs, 'h', ['help',
+                                            'env',
+                                            'pickle=',
+                                            ])
+
+        except getopt.GetoptError, v:
+            # print help information and exit:
+            print(v)
+            print(self.usage())
+            sys.exit(2)
+        
+        for o, a in self._opts:
+            if o in ('-h', '--help'):
+                print(self.usage())
+                sys.exit()
+            if o == '--env':
+                self._template.searchList().insert(0, os.environ)
+            if o == '--pickle':
+                if a == '-':
+                    unpickled = load(sys.stdin)
+                    self._template.searchList().insert(0, unpickled)
+                else:
+                    f = open(a)
+                    unpickled = load(f)
+                    f.close()
+                    self._template.searchList().insert(0, unpickled)
+
+    def usage(self):
+        return """Cheetah %(Version)s template module command-line interface
+
+Usage
+-----
+  %(scriptName)s [OPTION]
+
+Options
+-------
+  -h, --help                 Print this help information
+  
+  --env                      Use shell ENVIRONMENT variables to fill the
+                             $placeholders in the template.
+                             
+  --pickle             Use a variables from a dictionary stored in Python
+                             pickle file to fill $placeholders in the template.
+                             If  is - stdin is used: 
+                             '%(scriptName)s --pickle -'
+
+Description
+-----------
+
+This interface allows you to execute a Cheetah template from the command line
+and collect the output.  It can prepend the shell ENVIRONMENT or a pickled
+Python dictionary to the template's $placeholder searchList, overriding the
+defaults for the $placeholders.
+
+""" % {'scriptName': self._scriptName,
+       'Version': Version,
+       }
+
+# vim: shiftwidth=4 tabstop=4 expandtab

=== added directory 'cheetah/Templates'
=== added file 'cheetah/Templates/SkeletonPage.py'
--- cheetah/Templates/SkeletonPage.py	1970-01-01 00:00:00 +0000
+++ cheetah/Templates/SkeletonPage.py	2010-10-19 19:11:10 +0000
@@ -0,0 +1,272 @@
+
+
+"""A Skeleton HTML page template, that provides basic structure and utility methods.
+"""
+
+
+##################################################
+## DEPENDENCIES
+import sys
+import os
+import os.path
+from os.path import getmtime, exists
+import time
+import types
+import __builtin__
+from Cheetah.Version import MinCompatibleVersion as RequiredCheetahVersion
+from Cheetah.Version import MinCompatibleVersionTuple as RequiredCheetahVersionTuple
+from Cheetah.Template import Template
+from Cheetah.DummyTransaction import DummyTransaction
+from Cheetah.NameMapper import NotFound, valueForName, valueFromSearchList, valueFromFrameOrSearchList
+from Cheetah.CacheRegion import CacheRegion
+import Cheetah.Filters as Filters
+import Cheetah.ErrorCatchers as ErrorCatchers
+from Cheetah.Templates._SkeletonPage import _SkeletonPage
+
+##################################################
+## MODULE CONSTANTS
+try:
+    True, False
+except NameError:
+    True, False = (1==1), (1==0)
+VFFSL=valueFromFrameOrSearchList
+VFSL=valueFromSearchList
+VFN=valueForName
+currentTime=time.time
+__CHEETAH_version__ = '2.0rc6'
+__CHEETAH_versionTuple__ = (2, 0, 0, 'candidate', 6)
+__CHEETAH_genTime__ = 1139107954.3640411
+__CHEETAH_genTimestamp__ = 'Sat Feb  4 18:52:34 2006'
+__CHEETAH_src__ = 'src/Templates/SkeletonPage.tmpl'
+__CHEETAH_srcLastModified__ = 'Mon Oct  7 11:37:30 2002'
+__CHEETAH_docstring__ = 'Autogenerated by CHEETAH: The Python-Powered Template Engine'
+
+if __CHEETAH_versionTuple__ < RequiredCheetahVersionTuple:
+    raise AssertionError(
+      'This template was compiled with Cheetah version'
+      ' %s. Templates compiled before version %s must be recompiled.'%(
+         __CHEETAH_version__, RequiredCheetahVersion))
+
+##################################################
+## CLASSES
+
+class SkeletonPage(_SkeletonPage):
+
+    ##################################################
+    ## CHEETAH GENERATED METHODS
+
+
+    def __init__(self, *args, **KWs):
+
+        _SkeletonPage.__init__(self, *args, **KWs)
+        if not self._CHEETAH__instanceInitialized:
+            cheetahKWArgs = {}
+            allowedKWs = 'searchList namespaces filter filtersLib errorCatcher'.split()
+            for k, v in KWs.items():
+                if k in allowedKWs: cheetahKWArgs[k] = v
+            self._initCheetahInstance(**cheetahKWArgs)
+        
+
+    def writeHeadTag(self, **KWS):
+
+
+
+        ## CHEETAH: generated from #block writeHeadTag at line 22, col 1.
+        trans = KWS.get("trans")
+        if (not trans and not self._CHEETAH__isBuffering and not hasattr(self.transaction, '__call__')):
+            trans = self.transaction # is None unless self.awake() was called
+        if not trans:
+            trans = DummyTransaction()
+            _dummyTrans = True
+        else: _dummyTrans = False
+        write = trans.response().write
+        SL = self._CHEETAH__searchList
+        _filter = self._CHEETAH__currentFilter
+        
+        ########################################
+        ## START - generated method body
+        
+        write('\n')
+        _v = VFFSL(SL, "title", True) # '$title' on line 24, col 8
+        if _v is not None: write(_filter(_v, rawExpr='$title')) # from line 24, col 8.
+        write('\n')
+        _v = VFFSL(SL, "metaTags", True) # '$metaTags' on line 25, col 1
+        if _v is not None: write(_filter(_v, rawExpr='$metaTags')) # from line 25, col 1.
+        write(' \n')
+        _v = VFFSL(SL, "stylesheetTags", True) # '$stylesheetTags' on line 26, col 1
+        if _v is not None: write(_filter(_v, rawExpr='$stylesheetTags')) # from line 26, col 1.
+        write(' \n')
+        _v = VFFSL(SL, "javascriptTags", True) # '$javascriptTags' on line 27, col 1
+        if _v is not None: write(_filter(_v, rawExpr='$javascriptTags')) # from line 27, col 1.
+        write('\n\n')
+        
+        ########################################
+        ## END - generated method body
+        
+        return _dummyTrans and trans.response().getvalue() or ""
+        
+
+    def writeBody(self, **KWS):
+
+
+
+        ## CHEETAH: generated from #block writeBody at line 36, col 1.
+        trans = KWS.get("trans")
+        if (not trans and not self._CHEETAH__isBuffering and not hasattr(self.transaction, '__call__')):
+            trans = self.transaction # is None unless self.awake() was called
+        if not trans:
+            trans = DummyTransaction()
+            _dummyTrans = True
+        else: _dummyTrans = False
+        write = trans.response().write
+        SL = self._CHEETAH__searchList
+        _filter = self._CHEETAH__currentFilter
+        
+        ########################################
+        ## START - generated method body
+        
+        write('This skeleton page has no flesh. Its body needs to be implemented.\n')
+        
+        ########################################
+        ## END - generated method body
+        
+        return _dummyTrans and trans.response().getvalue() or ""
+        
+
+    def respond(self, trans=None):
+
+
+
+        ## CHEETAH: main method generated for this template
+        if (not trans and not self._CHEETAH__isBuffering and not hasattr(self.transaction, '__call__')):
+            trans = self.transaction # is None unless self.awake() was called
+        if not trans:
+            trans = DummyTransaction()
+            _dummyTrans = True
+        else: _dummyTrans = False
+        write = trans.response().write
+        SL = self._CHEETAH__searchList
+        _filter = self._CHEETAH__currentFilter
+        
+        ########################################
+        ## START - generated method body
+        
+        
+        ## START CACHE REGION: ID=header. line 6, col 1 in the source.
+        _RECACHE_header = False
+        _cacheRegion_header = self.getCacheRegion(regionID='header', cacheInfo={'type': 2, 'id': 'header'})
+        if _cacheRegion_header.isNew():
+            _RECACHE_header = True
+        _cacheItem_header = _cacheRegion_header.getCacheItem('header')
+        if _cacheItem_header.hasExpired():
+            _RECACHE_header = True
+        if (not _RECACHE_header) and _cacheItem_header.getRefreshTime():
+            try:
+                _output = _cacheItem_header.renderOutput()
+            except KeyError:
+                _RECACHE_header = True
+            else:
+                write(_output)
+                del _output
+        if _RECACHE_header or not _cacheItem_header.getRefreshTime():
+            _orig_transheader = trans
+            trans = _cacheCollector_header = DummyTransaction()
+            write = _cacheCollector_header.response().write
+            _v = VFFSL(SL, "docType", True) # '$docType' on line 7, col 1
+            if _v is not None: write(_filter(_v, rawExpr='$docType')) # from line 7, col 1.
+            write('\n')
+            _v = VFFSL(SL, "htmlTag", True) # '$htmlTag' on line 8, col 1
+            if _v is not None: write(_filter(_v, rawExpr='$htmlTag')) # from line 8, col 1.
+            write('''
+
+
+
+''')
+            self.writeHeadTag(trans=trans)
+            write('\n')
+            trans = _orig_transheader
+            write = trans.response().write
+            _cacheData = _cacheCollector_header.response().getvalue()
+            _cacheItem_header.setData(_cacheData)
+            write(_cacheData)
+            del _cacheData
+            del _cacheCollector_header
+            del _orig_transheader
+        ## END CACHE REGION: header
+        
+        write('\n')
+        _v = VFFSL(SL, "bodyTag", True) # '$bodyTag' on line 34, col 1
+        if _v is not None: write(_filter(_v, rawExpr='$bodyTag')) # from line 34, col 1.
+        write('\n\n')
+        self.writeBody(trans=trans)
+        write('''
+
+
+
+
+
+''')
+        
+        ########################################
+        ## END - generated method body
+        
+        return _dummyTrans and trans.response().getvalue() or ""
+        
+    ##################################################
+    ## CHEETAH GENERATED ATTRIBUTES
+
+
+    _CHEETAH__instanceInitialized = False
+
+    _CHEETAH_version = __CHEETAH_version__
+
+    _CHEETAH_versionTuple = __CHEETAH_versionTuple__
+
+    _CHEETAH_genTime = __CHEETAH_genTime__
+
+    _CHEETAH_genTimestamp = __CHEETAH_genTimestamp__
+
+    _CHEETAH_src = __CHEETAH_src__
+
+    _CHEETAH_srcLastModified = __CHEETAH_srcLastModified__
+
+    _mainCheetahMethod_for_SkeletonPage= 'respond'
+
+## END CLASS DEFINITION
+
+if not hasattr(SkeletonPage, '_initCheetahAttributes'):
+    templateAPIClass = getattr(SkeletonPage, '_CHEETAH_templateClass', Template)
+    templateAPIClass._addCheetahPlumbingCodeToClass(SkeletonPage)
+
+
+# CHEETAH was developed by Tavis Rudd and Mike Orr
+# with code, advice and input from many other volunteers.
+# For more information visit http://www.CheetahTemplate.org/
+
+##################################################
+## if run from command line:
+if __name__ == '__main__':
+    from Cheetah.TemplateCmdLineIface import CmdLineIface
+    CmdLineIface(templateObj=SkeletonPage()).run()
+
+

=== added file 'cheetah/Templates/SkeletonPage.tmpl'
--- cheetah/Templates/SkeletonPage.tmpl	1970-01-01 00:00:00 +0000
+++ cheetah/Templates/SkeletonPage.tmpl	2010-10-19 19:11:10 +0000
@@ -0,0 +1,44 @@
+##doc-module: A Skeleton HTML page template, that provides basic structure and utility methods.
+################################################################################
+#extends Cheetah.Templates._SkeletonPage
+#implements respond
+################################################################################
+#cache id='header'
+$docType
+$htmlTag
+
+
+
+#block writeHeadTag
+
+$title
+$metaTags 
+$stylesheetTags 
+$javascriptTags
+
+#end block writeHeadTag
+
+#end cache header
+#################
+
+$bodyTag
+
+#block writeBody
+This skeleton page has no flesh. Its body needs to be implemented.
+#end block writeBody
+
+
+
+
+
+

=== added file 'cheetah/Templates/_SkeletonPage.py'
--- cheetah/Templates/_SkeletonPage.py	1970-01-01 00:00:00 +0000
+++ cheetah/Templates/_SkeletonPage.py	2010-10-19 19:11:10 +0000
@@ -0,0 +1,215 @@
+# $Id: _SkeletonPage.py,v 1.13 2002/10/01 17:52:02 tavis_rudd Exp $
+"""A baseclass for the SkeletonPage template
+
+Meta-Data
+==========
+Author: Tavis Rudd ,
+Version: $Revision: 1.13 $
+Start Date: 2001/04/05
+Last Revision Date: $Date: 2002/10/01 17:52:02 $
+"""
+__author__ = "Tavis Rudd "
+__revision__ = "$Revision: 1.13 $"[11:-2]
+
+##################################################
+## DEPENDENCIES ##
+
+import time, types, os, sys
+
+# intra-package imports ...
+from Cheetah.Template import Template
+
+
+##################################################
+## GLOBALS AND CONSTANTS ##
+
+True = (1==1)
+False = (0==1)
+
+##################################################
+## CLASSES ##
+        
+class _SkeletonPage(Template):
+    """A baseclass for the SkeletonPage template"""
+
+    docType = ''
+    
+    # docType = ''
+        
+    title = ''
+    siteDomainName = 'www.example.com'
+    siteCredits = 'Designed & Implemented by Tavis Rudd'
+    siteCopyrightName = "Tavis Rudd"
+    htmlTag = ''
+    
+    def __init__(self, *args, **KWs):
+        Template.__init__(self, *args, **KWs)
+        self._metaTags = {'HTTP-EQUIV':{'keywords': 'Cheetah',
+                                        'Content-Type': 'text/html; charset=iso-8859-1',
+                                        }, 
+                    'NAME':{'generator':'Cheetah: The Python-Powered Template Engine'}
+                    }
+        # metaTags = {'HTTP_EQUIV':{'test':1234}, 'NAME':{'test':1234,'test2':1234} }
+        self._stylesheets = {}
+        # stylesheets = {'.cssClassName':'stylesheetCode'}
+        self._stylesheetsOrder = []
+        # stylesheetsOrder = ['.cssClassName',]
+        self._stylesheetLibs = {}
+        # stylesheetLibs = {'libName':'libSrcPath'}
+        self._javascriptLibs = {}
+        self._javascriptTags = {}
+        # self._javascriptLibs = {'libName':'libSrcPath'}
+        self._bodyTagAttribs = {}
+
+    def metaTags(self):
+        """Return a formatted vesion of the self._metaTags dictionary, using the
+        formatMetaTags function from Cheetah.Macros.HTML"""
+        
+        return self.formatMetaTags(self._metaTags)
+    
+    def stylesheetTags(self):
+        """Return a formatted version of the self._stylesheetLibs and
+        self._stylesheets dictionaries.  The keys in self._stylesheets must
+        be listed in the order that they should appear in the list
+        self._stylesheetsOrder, to ensure that the style rules are defined in
+        the correct order."""
+        
+        stylesheetTagsTxt = ''
+        for title, src in self._stylesheetLibs.items():
+            stylesheetTagsTxt += '\n'
+
+        if not self._stylesheetsOrder:
+            return stylesheetTagsTxt
+        
+        stylesheetTagsTxt += '\n'
+
+        return stylesheetTagsTxt
+
+    def javascriptTags(self):
+        """Return a formatted version of the javascriptTags and
+        javascriptLibs dictionaries.  Each value in javascriptTags
+        should be a either a code string to include, or a list containing the
+        JavaScript version number and the code string. The keys can be anything.
+        The same applies for javascriptLibs, but the string should be the
+        SRC filename rather than a code string."""
+        
+        javascriptTagsTxt = []
+        for key, details in self._javascriptTags.iteritems():
+            if not isinstance(details, (list, tuple)):
+                details = ['', details]
+                
+            javascriptTagsTxt += ['\n']
+
+
+        for key, details in self._javascriptLibs.iteritems():
+            if not isinstance(details, (list, tuple)):
+                details = ['', details]
+
+            javascriptTagsTxt += ['\n']
-
-
-        for key, details in self._javascriptLibs.items():
-            if type(details) not in (types.ListType, types.TupleType):
-                details = ['',details]
-
-            javascriptTagsTxt += ['