Plugins
*******


Introduction
============

Nagelfar allows you to set up a list of plugins that can hook up and
affect the checks in different stages.

A plugin can be used for things like:

* Enforce local rules

* Handle checks for custom commands that cannot be handled within
  Nagelfar's syntax tokens

* Annotate or ignore constructs in e.g. legacy code that cannot be
  changed

* Collect statistics, e.g. a call graph

See further Examples below.


Generic rules
-------------

A plugin is a Tcl script file that must start with the verbatim
sequence "##Nagelfar Plugin :". A plugin is sourced and used in its
own safe interpreter and thus have free access to its own global
space. Hookup points are defined by declaring specifically named
procedures as specified below, and apart from those, a plugin can
define and do whatever within the limits of a safe interpreter.

In addition to the standard safe interpreter environment, a plugin has
access to "stdout" as well.

Note that backslash-newlines are always removed at an early stage in
Nagelfar, so when hooks receive "unparsed" data, those have been
removed.


Usage
-----

To activate a plugin, use "-plugin <plugin>" on the command line.
Repeat to load additional plugins in the specified order.

Nagelfar searches for plugins (looking for names <plugin> and
<plugin>.tcl) in the following places:

* . (Current directory)

* ./plugins

* <nagelfar source>/../..

* <nagelfar source>/../../plugins

* <nagelfar source>/../plugins

* "-pluginpath <dir>" Option

Currently the plugin cannot be selected from the GUI.


Result of plugin procedures
---------------------------

Each hookup procedure returns a list with an even number of elements.
These are interpreted as keyword-value pairs, with the following
keywords allowed.

* replace : The value is used to replace the incoming value for
  further processing.
     If multiple plugins use replace only the last value is effective.

* comment : The value is fed through inline comment parsing to affect
  surroundings.

* error   : The value produces an error message.

* warning : The value produces a warning message.

* note    : The value produces a note message.

To do nothing, return an empty list.

Messages will be reported on the first line of whatever the hook gets
to look at.


Information dictionary
----------------------

Each hook procedure receives an information dictionary as one
argument. It currently has at least these elements:

* namespace : Current namespace

* caller    : Current procedure

* file      : Current file

* firstpass : True during first scan of code. False during full check.

* vars      : Dictionary where each key is a known variable in current
  scope.

* cmds      : List of command names saved in the syntax database.


Hooks
=====


Finalizing checking
-------------------

"proc finalizePlugin {} { }"

If this procedure is declared, it is called at the end of checking.

The return value from finalizePlugin may only contain messages.


Raw Body Hook
-------------

"proc bodyRaw {stmt info} { }"

If declared, this receives each script body unparsed (any backslash-
newline is removed). The info dictionary includes the element "subst"
to indicate if the body is in a command substitution.


Raw Statement Hook
------------------

"proc statementRaw {stmt info} { }"

If declared, this receives each statement unparsed (any backslash-
newline is removed).


Statement Words Hook
--------------------

"proc statementWords {words info} { }"

If declared, this receives each statement split into a list of words
but otherwise unprocessed/unsubstituted. Things like quotes and braces
are left in the words.

Many checks can be done in a simple way here since you have direct
access to the command word and the number of arguments.


Raw Expression Hook
-------------------

"proc earlyExpr {exp info} { }"

If declared, this receives any expression unparsed.


Late Expression Hook
--------------------

"proc lateExpr {exp info} { }"

If declared, this receives any expression after comments are removed
and all variable or command substitutions have been replaced by
"${_____}". It is still basically the same expression and this allows
a handler that knows fewer syntax rules.


Variable Write Hook
-------------------

"proc varWrite {var info} { }"

If declared, this receives any variable written to.


Variable Read Hook
------------------

"proc varRead {var info} { }"

If declared, this receives any variable read from.


Raw Line Hook
-------------

"proc lineRaw {line info} { }"

If declared, this receives each line unaltered while a file is
initially read.

In the info dictionary, namespace, caller, and firstpass are not
relevant.


Unknown Command Hook
--------------------

"proc unknownCommand {cmd info} { }"

If declared, this receives any encountered unknown command.


Write Header Hook
-----------------

"proc writeHeader {} {}"

If declared, called when writing a file with -header. Every Inline-
comment returned is appended to the file.


Read Inline Comments
--------------------

"proc syntaxComment {type opts} {}"

If declared, receives any "##nagelfar <type> <opts...>" in the input.
May return true to disable default action, e.g. if the type is plugin
specific.


Examples
========


Call Graph
----------

   ##Nagelfar Plugin : Create a call graph
   proc statementWords {words info} {
       set caller [dict get $info caller]
       set callee [lindex $words 0]
       if {$caller ne "" && $callee ne ""} {
           array set ::callGraph [list "$caller -> $callee" 1]
       }
       return
   }
   proc finalizePlugin {} {
       foreach item [lsort -dictionary [array names ::callGraph]] {
           puts "Call: $item"
       }
       return
   }


Ignore a command
----------------

   ##Nagelfar Plugin : Ignore mugg command
   proc statementRaw {stmt info} {
       set res {}
       if {[string match "mugg *" $stmt]} {
           lappend res replace {}
       }
       return $res
   }


Handle known side effect
------------------------

   ##Nagelfar Plugin : Handle known side effect
   proc statementWords {words info} {
       set res {}
       # The command "mugg" sets a variable in the caller
       if {[lindex $words 0] eq "mugg"} {
           lappend res comment
           lappend res "##nagelfar variable gurka"
       }
       return $res
   }


Forbid operator
---------------

   ##Nagelfar Plugin : Forbid operator
   proc lateExpr {exp info} {
       if {[string match "* eq *" $exp]} {
           return [list error "Operator \"eq\" is forbidden here"]
       }
       return {}
   }


Check coding standard
---------------------

   ##Nagelfar Plugin : Check coding rule
   proc statementWords {words info} {
       set res {}
       # We require space around ! in if { ! []}
       if {[lindex $words 0] eq "if"} {
           set e [lindex $words 1]
           if {[regexp {\{(\s*)!(\s*)\[} $e -> pre post]} {
              if {$pre ne " " || $post ne " "} {
                  lappend res warning
                  lappend res "Not (!) should be surrounded by one space"
              }
          }
       }
       return $res
   }


Allow custom operator
---------------------

   ##Nagelfar Plugin : Allow custom operator
   proc lateExpr {exp info} {
       # Just replace it with something further processing recognizes
       set exp [string map {{ my_cool_bin_op } { eq }} $exp]
       return [list replace $exp]
   }


Look for operator usage
-----------------------

   ##Nagelfar Plugin : Operator with string literal
   # In the wake of TIP#461, help looking for things that can become a problem.
   proc lateExpr {exp info} {
       # Any comparison operator vs literal string give a note
       # The regexp could be more precise of course.
       if {[regexp {(!=|==|<|<=|>|>=)\s*\"} $exp -> op]} {
           return [list note "Operator \"$op\" used with string literal"]
       }
       if {[regexp {\"\s*(!=|==|<|<=|>|>=)} $exp -> op]} {
           return [list note "Operator \"$op\" used with string literal"]
       }
       return ""
   }


Handle special syntax
---------------------

   ##Nagelfar Plugin : Handle special syntax
   proc statementWords {words info} {
       set res {}
       # We are only interested in calls to "mugg"
       if {[lindex $words 0] ne "mugg"} {
           return $res
       }
       # If a command has varying syntax depending on contents it can be handled,
       # compare e.g. with a complex command like "if".
       # In this example, only 1 or 5 arguments are allowed, which could
       # also be expressed directly with the syntax string "1: x : 5"
       lappend res comment
       if {[llength $words] == 6} {
           lappend res "##nagelfar syntax mugg x x x x x"
       } else {
           lappend res "##nagelfar syntax mugg x"
       }
       return $res
   }


Check for unused globals
------------------------

   ##Nagelfar Plugin : Check for unused globals
   set ::data {}
   proc statementWords {words info} {
       if {[lindex $words 0] ne "global"} return
       set caller [dict get $info caller]
       foreach var [lrange $words 1 end] {
           dict set ::data $caller $var 1
       }
       return
   }
   proc varWrite {var info} {
       set caller [dict get $info caller]
       dict unset ::data $caller $var
       return
   }
   proc varRead {var info} {
       set caller [dict get $info caller]
       dict unset ::data $caller $var
       return
   }
   proc finalizePlugin {} {
       set res {}
       foreach caller [dict keys $::data] {
           foreach var [dict keys [dict get $::data $caller]] {
               lappend res warning "Unused global '$var' in proc '$caller'"
           }
       }
       lappend res note "Globals checked by plugin"
       return $res
   }


Sqlite code
-----------

In code like this, using the sqlite3 package:

   db eval { SELECT rowid,name,start FROM SQLYSTUFF } {
       list $rowid $name
   }
   db eval {UPDATE tasks SET user = $u, initial = 320 WHERE rowid = $g}

Nagelfar cannot know that rowid and name are existing variables and
will give an error. A plugin can parse the SQL and provide this info.

Similarly Nagelfar does not know that the SQL code can contain
variable references. Checking those can also be done.

   ##Nagelfar Plugin : Sqlite handler
   proc statementWords {words info} {
       # We are only interested in calls to "db eval <sql> ?<code>?"
       if {[lindex $words 0] ne "db"} return
       if {[lindex $words 1] ne "eval"} return
       if {[llength $words] < 3} return
       set sql [lindex $words 2]
       set res {}
       # Looking for variable reads
       foreach {_ var} [regexp -all -inline {[$:](\w+)} $sql] {
           if {![dict exists $info vars $var]} {
               lappend res warning
               lappend res "Unknown variable '$var'"
           }
       }
       # Simple "parser" assuming a certain format to detect variables set
       if {[llength $words] == 4} {
           if {[regexp {SELECT (.*) FROM} [lindex $words 2] -> vars]} {
               foreach var [regexp -all -inline {\w+} $vars] {
                   lappend res comment
                   lappend res "##nagelfar variable $var"
               }
           }
       }
       return $res
   }


Namespace eval check
--------------------

Detect creative writing in namespace eval code.

   ##Nagelfar Plugin : Namespace eval check
   proc statementWords {words info} {
       set caller [dict get $info caller]
       # Code in proc is not interesting
       if {$caller ne ""} return
       set ns [dict get $info namespace]
       # Global is not interesting
       if {$ns eq "" || $ns eq "::"} return
       set cmd [lindex $words 0]
       if {$cmd eq "variable"} {
           foreach {var _} [lindex $words 1 end] {
               set ::known(${ns}::$var) 1
           }
       }
       return
   }
   proc varWrite {var info} {
       set caller [dict get $info caller]
       # Code in proc is not interesting
       if {$caller ne ""} return
       set ns [dict get $info namespace]
       # Global is not interesting
       if {$ns eq "" || $ns eq "::"} return
       if {![info exists ::known(${ns}::$var)]} {
           return [list warning "Writing $var without variable call"]
       }
   }


Deprecation Notice
------------------

Add deprecation warning to proc, and save info to Header.

   ##Nagelfar Plugin : Deprecation Notice
   set ::deprecated {}
   proc syntaxComment {type opts} {
       if {$type eq "deprecated"} {
           lappend ::deprecated [lindex $opts 0]
           return true
       }
       return false
   }
   proc statementWords {words info} {
       if {[lindex $words 0] in $::deprecated} {
           return [list warning "[lindex $words 0] is deprecated"]
       }
       return {}
   }
   proc writeHeader {} {
      set res {}
      foreach cmd [lsort -unique $::deprecated] {
          lappend res "##nagelfar deprecated $cmd"
      }
      return $res
   }


Detect proc overwrite
---------------------

Warn if a procedure is defined twice.

   ##Nagelfar Plugin : Detect proc redefinition
   proc statementWords {words info} {
       set res {}
       # Skip the first pass
       if {[dict get $info firstpass]} {
           return $res
       }
       # We are only interested in calls to "proc"
       if {[lindex $words 0] ne "proc"} {
           return $res
       }
       # Quick and dirty namespace resolve. Might need work.
       set ns [dict get $info namespace]
       set name [lindex $words 1]
       if {[string match ::* $name]} {
           set fullName $name
       } else {
           set fullName ${ns}::$name
       }
       if {[info exists ::seen($fullName)]} {
           lappend res warning
           lappend res "Redefined proc \"$name\""
       } else {
           set ::seen($fullName) [dict get $info file]
       }
       return $res
   }


Handle unknown commands
-----------------------

If a command is unknown, suggest a similar command from the list of
known commands.

   ##Nagelfar Plugin : Suggest possible fixes
   proc unknownCommand {cmd info} {
       set res {}
       set cmds [dict get $info cmds]
       set candidates [lsearch -glob -all -inline $cmds "*::$cmd"]
       set candidates [lsort -dictionary $candidates]
       if {[llength $candidates] == 0} {
           return $res
       }
       set fixSuggestion \"[join $candidates "\", \""]\"

       if {[llength $candidates] == 1} {
           set msg "Perhaps you meant: $fixSuggestion"
       } else { # [llength $candidates] > 1
           set msg "Perhaps you meant one of: $fixSuggestion"
       }
       lappend res "warning"
       lappend res "$msg"
       return $res
   }
