README
¶
gsx
An HTML manipulation toolkit
gsx is a lightweight html toolkit for parsing, manipulating and generating HTML.
Examples
Examples can be found in the ./examples directory
When should I use this?
This is not a "browser-grade" HTML parser, but it is close!
Specifically, the tokenizer is spec compliant and passes all the html5lib tokenizer tests.
So gsx will accept any valid HTML "construct" like numeric & named character references and void elements.
However, the tree-builder does not follow the spec. This was done on purpose. A spec compliant tree-builder may restructure your markup for multitude of reasons: badly nested tags, child elements that don't conform to the content model of their parent, missing end tags etc ... The tree-builder in this module takes a simpler approach: it will parse any well-balanced HTML and output a tree that corresponds to that markup, exactly as written.
So this library will work well when you are parsing the output of HTML-generating tools like SSGs or markdown parser. Tools like these don't forget to add end tags :)
On the other hand, parsing random web content is more of a gamble.
For example, many sites rely on the fact that you do not need to close your <p> tags.
This library will fail on such markup.
TLDR; If your HTML looks like well-formed XML if you squint, this library's HTML parser is for you.
Documentation
¶
Overview ¶
Example (Attributes) ¶
root, err := Parse(`<input name="foo" value="bar" readonly>`)
if err != nil {
panic(err)
}
input := root.Descendants().Filter(`input[name=foo]`).First()
{
v, ok := input.Attr("value")
fmt.Printf("value=%q/%v\n", v, ok)
}
{
v, ok := input.Attr("readonly")
fmt.Printf("readonly=%q/%v\n", v, ok)
}
{
v, ok := input.Attr("blah")
fmt.Printf("blah=%q/%v\n", v, ok)
}
Output: value="bar"/true readonly=""/true blah=""/false
Example (Generating) ¶
form := Form(
Attrs{"method": "GET"},
Input(Attrs{"value": "hello", "type": "text", "readonly": ""}),
Button(nil, Text("Submit")),
)
fmt.Println(form.Html())
Output: <form method="GET"><input readonly="" type="text" value="hello"><button>Submit</button></form>
Example (Manipulating) ¶
root, err := Parse(`<html><body>hello<p class="hello">REMOVE ME</p></body></html>`)
if err != nil {
panic(err)
}
target := root.Descendants().Filter(`.hello`).First()
target.Detach()
fmt.Println(root.Html())
Output: <html><body>hello</body></html>
Example (Parsing) ¶
root, err := Parse("<h1>Hello, <i>world!</i></h1>")
if err != nil {
panic(err)
}
fmt.Println(root.Html())
Output: <h1>Hello, <i>world!</i></h1>
Example (Selecting) ¶
root, err := Parse(`
<ul>
<li>Foo</li>
<li>Bar</li>
</ul>
<div>
<span>One</span>
<span>Two</span>
</div>
`)
if err != nil {
panic(err)
}
fmt.Println("===")
base := root.Descendants()
liElements := base.Filter("ul > li")
spanElements := base.Filter("div > span")
for el := range liElements.Chain(spanElements) {
fmt.Println(el.TextContent())
}
fmt.Println("===")
// The selections were broken up to emphasise they are composable.
// But you could also do:
for el := range root.Descendants().Filter("ul > li").Chain(root.Descendants().Filter("div > span")) {
fmt.Println(el.TextContent())
}
Output: === Foo Bar One Two === Foo Bar One Two
Index ¶
- Variables
- type Attrs
- type Node
- func A(attributes Attrs, children ...*Node) *Node
- func Abbr(attributes Attrs, children ...*Node) *Node
- func Address(attributes Attrs, children ...*Node) *Node
- func Area(attributes Attrs, children ...*Node) *Node
- func Article(attributes Attrs, children ...*Node) *Node
- func Aside(attributes Attrs, children ...*Node) *Node
- func Audio(attributes Attrs, children ...*Node) *Node
- func B(attributes Attrs, children ...*Node) *Node
- func Base(attributes Attrs, children ...*Node) *Node
- func Bdi(attributes Attrs, children ...*Node) *Node
- func Bdo(attributes Attrs, children ...*Node) *Node
- func Blockquote(attributes Attrs, children ...*Node) *Node
- func Body(attributes Attrs, children ...*Node) *Node
- func Br(attributes Attrs, children ...*Node) *Node
- func Button(attributes Attrs, children ...*Node) *Node
- func Canvas(attributes Attrs, children ...*Node) *Node
- func Caption(attributes Attrs, children ...*Node) *Node
- func Cite(attributes Attrs, children ...*Node) *Node
- func Code(attributes Attrs, children ...*Node) *Node
- func Col(attributes Attrs, children ...*Node) *Node
- func Colgroup(attributes Attrs, children ...*Node) *Node
- func CollectFragment(iter Selection) *Node
- func Data(attributes Attrs, children ...*Node) *Node
- func Datalist(attributes Attrs, children ...*Node) *Node
- func Dd(attributes Attrs, children ...*Node) *Node
- func Del(attributes Attrs, children ...*Node) *Node
- func Details(attributes Attrs, children ...*Node) *Node
- func Dfn(attributes Attrs, children ...*Node) *Node
- func Dialog(attributes Attrs, children ...*Node) *Node
- func Div(attributes Attrs, children ...*Node) *Node
- func Dl(attributes Attrs, children ...*Node) *Node
- func Doctype() *Node
- func Dt(attributes Attrs, children ...*Node) *Node
- func Em(attributes Attrs, children ...*Node) *Node
- func Embed(attributes Attrs, children ...*Node) *Node
- func Fieldset(attributes Attrs, children ...*Node) *Node
- func Figcaption(attributes Attrs, children ...*Node) *Node
- func Figure(attributes Attrs, children ...*Node) *Node
- func Footer(attributes Attrs, children ...*Node) *Node
- func Form(attributes Attrs, children ...*Node) *Node
- func Fragment(nodes ...*Node) *Node
- func H1(attributes Attrs, children ...*Node) *Node
- func H2(attributes Attrs, children ...*Node) *Node
- func H3(attributes Attrs, children ...*Node) *Node
- func H4(attributes Attrs, children ...*Node) *Node
- func H5(attributes Attrs, children ...*Node) *Node
- func H6(attributes Attrs, children ...*Node) *Node
- func Head(attributes Attrs, children ...*Node) *Node
- func Header(attributes Attrs, children ...*Node) *Node
- func Hgroup(attributes Attrs, children ...*Node) *Node
- func Hr(attributes Attrs, children ...*Node) *Node
- func Html(attributes Attrs, children ...*Node) *Node
- func I(attributes Attrs, children ...*Node) *Node
- func Iframe(attributes Attrs, text string) *Node
- func Img(attributes Attrs, children ...*Node) *Node
- func Input(attributes Attrs, children ...*Node) *Node
- func Ins(attributes Attrs, children ...*Node) *Node
- func Kbd(attributes Attrs, children ...*Node) *Node
- func Label(attributes Attrs, children ...*Node) *Node
- func Legend(attributes Attrs, children ...*Node) *Node
- func Li(attributes Attrs, children ...*Node) *Node
- func Link(attributes Attrs, children ...*Node) *Node
- func Main(attributes Attrs, children ...*Node) *Node
- func Map(attributes Attrs, children ...*Node) *Node
- func Mark(attributes Attrs, children ...*Node) *Node
- func Menu(attributes Attrs, children ...*Node) *Node
- func Meta(attributes Attrs, children ...*Node) *Node
- func Meter(attributes Attrs, children ...*Node) *Node
- func Nav(attributes Attrs, children ...*Node) *Node
- func Noscript(attributes Attrs, text string) *Node
- func Object(attributes Attrs, children ...*Node) *Node
- func Ol(attributes Attrs, children ...*Node) *Node
- func Optgroup(attributes Attrs, children ...*Node) *Node
- func Option(attributes Attrs, children ...*Node) *Node
- func Output(attributes Attrs, children ...*Node) *Node
- func P(attributes Attrs, children ...*Node) *Node
- func Parse(html string) (*Node, error)
- func Picture(attributes Attrs, children ...*Node) *Node
- func Pre(attributes Attrs, children ...*Node) *Node
- func Progress(attributes Attrs, children ...*Node) *Node
- func Q(attributes Attrs, children ...*Node) *Node
- func RawText(text string) *Node
- func Rp(attributes Attrs, children ...*Node) *Node
- func Rt(attributes Attrs, children ...*Node) *Node
- func Ruby(attributes Attrs, children ...*Node) *Node
- func S(attributes Attrs, children ...*Node) *Node
- func Samp(attributes Attrs, children ...*Node) *Node
- func Script(attributes Attrs, text string) *Node
- func Search(attributes Attrs, children ...*Node) *Node
- func Section(attributes Attrs, children ...*Node) *Node
- func Select(attributes Attrs, children ...*Node) *Node
- func Selectedcontent(attributes Attrs, children ...*Node) *Node
- func Slot(attributes Attrs, children ...*Node) *Node
- func Small(attributes Attrs, children ...*Node) *Node
- func Source(attributes Attrs, children ...*Node) *Node
- func Span(attributes Attrs, children ...*Node) *Node
- func Strong(attributes Attrs, children ...*Node) *Node
- func Style(attributes Attrs, text string) *Node
- func Sub(attributes Attrs, children ...*Node) *Node
- func Summary(attributes Attrs, children ...*Node) *Node
- func Sup(attributes Attrs, children ...*Node) *Node
- func Table(attributes Attrs, children ...*Node) *Node
- func Tag(name string, attributes Attrs, children ...*Node) *Node
- func Tbody(attributes Attrs, children ...*Node) *Node
- func Td(attributes Attrs, children ...*Node) *Node
- func Template(attributes Attrs, children ...*Node) *Node
- func Text(text string) *Node
- func Textarea(attributes Attrs, text string) *Node
- func Tfoot(attributes Attrs, children ...*Node) *Node
- func Th(attributes Attrs, children ...*Node) *Node
- func Thead(attributes Attrs, children ...*Node) *Node
- func Time(attributes Attrs, children ...*Node) *Node
- func Title(attributes Attrs, text string) *Node
- func Tr(attributes Attrs, children ...*Node) *Node
- func Track(attributes Attrs, children ...*Node) *Node
- func U(attributes Attrs, children ...*Node) *Node
- func Ul(attributes Attrs, children ...*Node) *Node
- func Var(attributes Attrs, children ...*Node) *Node
- func Video(attributes Attrs, children ...*Node) *Node
- func Wbr(attributes Attrs, children ...*Node) *Node
- func (self *Node) Ancestors() Selection
- func (self *Node) Append(children ...*Node)
- func (self *Node) Attr(name string) (string, bool)
- func (self *Node) Children() Selection
- func (self *Node) Descendants() Selection
- func (self *Node) Detach()
- func (self *Node) FirstChild() *Node
- func (self *Node) Following() Selection
- func (self *Node) Html() string
- func (self *Node) InsertAfter(siblings ...*Node)
- func (self *Node) InsertBefore(siblings ...*Node)
- func (self *Node) Kind() NodeKind
- func (self *Node) LastChild() *Node
- func (self *Node) Name() string
- func (self *Node) Next() *Node
- func (self *Node) Parent() *Node
- func (self *Node) Preceding() Selection
- func (self *Node) Prepend(children ...*Node)
- func (self *Node) Previous() *Node
- func (self *Node) RemoveAttr(name string)
- func (self *Node) ReplaceChildren(newChildren ...*Node)
- func (self *Node) ReverseChildren() Selection
- func (self *Node) Serialize(sink io.Writer) error
- func (self *Node) SetAttr(name, value string)
- func (self *Node) String() string
- func (self *Node) TextContent() string
- type NodeKind
- type ParserError
- type Selection
- func (self Selection) Chain(other Selection) Selection
- func (self Selection) Collect() []*Node
- func (self Selection) Exclude(selector string) Selection
- func (self Selection) Filter(selector string) Selection
- func (self Selection) First() *Node
- func (self Selection) Last() *Node
- func (self Selection) Len() int
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ex: </span> ErrEndTagWithoutCorrespondingStartTag = "no-matching-start-tag" // ex: <div> ErrUnclosedStartTag = "unclosed-start-tag" // ex: </ link> // ex: </ link/> ErrVoidElementAsEndTag = "void-element-as-end-tag" // ex: <div /> ErrNonVoidElementStartTagWithTrailingSolidus = "non-void-element-with-trailing-solidus" )
The ParserError.Code variants that Parse might return.
Functions ¶
This section is empty.
Types ¶
type Node ¶
type Node struct {
// contains filtered or unexported fields
}
An object within an HTML document.
There are 5 types of nodes (see NodeKind). Nodes can have at most one parent, zero or more siblings and zero or more children.
func Blockquote ¶
Returns a new '<blockquote>' node.
func CollectFragment ¶
Like Fragment, but accepts a Node iterator instead.
func Figcaption ¶
Returns a new '<figcaption>' node.
func Fragment ¶
Returns a NodeKindFragment node with the given children.
Fragment nodes allow you to group multiple nodes without adding an extra wrapper element.
If/when you append a fragment node into another node, the fragment node "disapears" and its children get appended instead. Same thing applies for inserting.
Fragment nodes never have a [Parent], so calling [InsertBefore] or [InsertAfter] with a fragment node as a receiver will panic.
func Parse ¶
Parses the given HTML fragment.
Parsing is done in two phases: tokenization and tree construction:
- The tokenization phase is spec compliant, so it does not fail hard, instead errors are recovered from in a spec-compliant way. The input will still be converted to tokens and emitted to the next phase.
- The tree construction phase implements a very small subset of the spec, and fails at the first ParserError, which is returned to the caller.
func RawText ¶
Returns a NodeKindText node with the given contents.
Unlike Text, text passed in to this function will be serialized verbatim.
This makes it useful for inlining svgs for example.
func Selectedcontent ¶
Returns a new '<selectedcontent>' node.
func Text ¶
Returns a NodeKindText node with the given contents.
The text will be properly escaped when serializing to HTML.
func (*Node) Append ¶
Inserts children after this node's last child.
Inserting a node of kind NodeKindFragment inserts its children instead.
Only NodeKindFragment and NodeKindElement nodes can have children, so this method panics if the receiver's kind is something else.
func (*Node) Attr ¶
Returns the value of the attribute with the given name.
The name comparison is done case insensitively.
If this is no such attribute, or if the receiver isn't a NodeKindElement the returned boolean is set to false.
func (*Node) Descendants ¶
Returns an iterator of descendant nodes.
func (*Node) Detach ¶
func (self *Node) Detach()
Orphans this node by detaching it from its parent and siblings.
Children are not affected.
This has no effect on nodes of kind NodeKindFragment.
func (*Node) FirstChild ¶
Returns this node's first child, if any.
func (*Node) Html ¶
Returns the HTML representation of the node.
This is the same thing as calling Node.Serialize with a strings.Builder.
func (*Node) InsertAfter ¶
Inserts siblings after this node
Inserting a node of kind NodeKindFragment inserts its children instead.
This method panics if the receiver does not have a parent.
func (*Node) InsertBefore ¶
Inserts siblings before this node
Inserting a node of kind NodeKindFragment inserts its children instead.
This method panics if the receiver does not have a parent.
func (*Node) Name ¶
Return's the node's name.
For NodeKindElement, this returns the tag name. For NodeKindDoctype this returns the doctype name For all other node kinds, this returns an empty string
func (*Node) Parent ¶
Returns this node's parent, if any.
Nodes of kind NodeKindFragment never have a parent.
func (*Node) Prepend ¶
Inserts children before this node's first child
Inserting a node of kind NodeKindFragment inserts its children instead.
Only NodeKindFragment and NodeKindElement nodes can have children, so this method panics if the receiver's kind is something else.
func (*Node) RemoveAttr ¶
Removes the attribute with the given name from the node.
func (*Node) ReplaceChildren ¶
Replaces the existing children of the receiver with a new set of children.
Only NodeKindFragment and NodeKindElement nodes can have children, so this method panics if the receiver's kind is something else.
func (*Node) ReverseChildren ¶
Returns an iterator of child nodes in reverse order.
func (*Node) SetAttr ¶
Sets the value of the given attribute on the node.
If the element already had an attribute by this name, it is overriden. Attribute names are case-insensitive, so this rule applies even if the existing attribute was inserted using a different case.
func (*Node) TextContent ¶
Returns the text content of this node and its descendants.
If the receiver is a NodeKindComment, this returns the comment contents.
Returns an empty string if the receiver is a NodeKindDoctype.
type NodeKind ¶
type NodeKind uint8
The type of the node.
const ( // An HTML tag NodeKindElement NodeKind = iota // Text within an HTML element. NodeKindText // An HTML comment (e.g. `<!-- blah -->`) NodeKindComment // A doctype element (e.g. `<!DOCTYPE html>`) NodeKindDoctype // A synthetic node that exists purely to contain the other nodes. NodeKindFragment )
func (NodeKind) String ¶
NodeKind implements the fmt.Stringer interface.
type ParserError ¶
type ParserError struct {
// One of:
// - [ErrEndTagWithoutCorrespondingStartTag]
// - [ErrUnclosedStartTag]
// - [ErrVoidElementAsEndTag]
// - [ErrNonVoidElementStartTagWithTrailingSolidus]
Code string
// The byte offset into [ParserError.Markup] at which the error occurred
Offset int
// The HTML source.
// The Parser will normalize the HTML input according to the speck, so this might be slightly different from what you provided to [Parse]
Markup string
}
An error that occured while parsing HTML
func (*ParserError) Error ¶
func (self *ParserError) Error() string
type Selection ¶
A Selection wraps an iterator of nodes. A nil Selection is empty.
Some methods on this struct accept a css selector. If the selector is invalid, this function panics.
Such methods support a subset of regular CSS selectors. They will accept what the spec calls a complex selector. However, pseudo-elements, pseudo-classes (e.g. `:first-child`, `:hover`) and namespaces are not supported.
Here are some examples:
// Matches any node. `*` // Matches all <div> nodes. `div` // Matches all <button> nodes that are direct children of a <div> element. `div > button` // Matches all <div> nodes whose "data-foo" attribute starts with "bar". `div[data-foo^="bar"]`
Type, universal, class, ID and attribute selectors are recognized. For attribute selectors though, the case-sensitivity flags are not supported.
func (Selection) Chain ¶
Returns a new selection that first yields the elements from receiver and then the elements from the other selection.
In other words, it links to Selections together in a chain.
func (Selection) Exclude ¶
Returns a new selection that will only yield elements that do not match selector.
Css selectors only match elements, so the resulting selection will only yield NodeKindElement nodes.
func (Selection) Filter ¶
Returns a new selection that will only yield elements that match selector.
Css selectors only match elements, so the resulting selection will only yield NodeKindElement nodes.
func (Selection) First ¶
Returns the first element of the selection.
This will be nil if the selection is empty.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
internal
|
|
|
cmd/generate_char_ref_map
command
|
|
|
cmd/generate_tags
command
|
|
|
queue
Paackage queue implements a growable queue implemented using a circlar buffer
|
Paackage queue implements a growable queue implemented using a circlar buffer |