Hi Peter, When I've run into these sorts of situations, I've found delegating XML generation to the database to be faster and acceptable. Before going further into the details, what's your expected payload size, i.e. how many records and how many attributes per record, and how big are these attributes? Secondly, do you have the option to create rows of xml to return directly from the database? I'm unsure how you're profiling your code, there might be lower level methods you can use, but it's difficult to assume without seeing what/how you're doing it. Kind regards, Vinod On Thu, May 18, 2023 at 6:21 PM Peter Hickman via ruby-talk < ruby-talk@ml.ruby-lang.org> wrote:
Thanks for the input but I have not been able to improve the performance of the generation by that much. The output did contain repeated sections that I have cached to speed things up but the variability of the runtime is quite insane. The average is 9.729 seconds which itself is not good but there isn't even consistency in repeated calls with the same data
So perhaps there is another issue
Thanks for your help, it has been an interesting rabbit hole
On Thu, 18 May 2023 at 13:35, Ruby users (Avocadostore) < support@avocadostore.zendesk.com> wrote:
##- Bitte geben Sie Ihre Antwort über dieser Zeile ein. -##
Sie sind in dieser Supportanfrage (2001706) auf CC gesetzt. Antworten Sie auf diese E-Mail, um Kommentare zur Anfrage hinzuzufügen.
*Ruby users*
18. Mai 2023, 14:35 MESZ
For posterity, I've put the benchmark code and results from my previous email here:
https://gist.github.com/flavorjones/4875b772b7517185913bded8e18b1ed5
In case I didn't make it clear, using Nokogiri's Document API is faster than Builder and (unless you have benchmarks showing something else) I still recommend using it over rolling your own templating solution.
On Thu, May 18, 2023 at 8:24 AM Peter Hickman via ruby-talk < ruby-talk@ml.ruby-lang.org> wrote:
So far building it by hand is no faster than using builder. Which I'm going to see as a plus for builder because it sure is easier to read
I think that the real issue is building a big string piecemeal is where the time is being spent
-K0GVP]
______________________________________________ ruby-talk mailing list -- ruby-talk@ml.ruby-lang.org To unsubscribe send an email to ruby-talk-leave@ml.ruby-lang.org ruby-talk info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-talk.ml.ruby-lang.org...
*Peter Hickman*
18. Mai 2023, 14:24 MESZ
So far building it by hand is no faster than using builder. Which I'm going to see as a plus for builder because it sure is easier to read
I think that the real issue is building a big string piecemeal is where the time is being spent
-K0GVP]
*Ruby users*
18. Mai 2023, 14:18 MESZ
If you want to optimize for speed, I'd recommend using Nokogiri, but not the Nokogiri::XML::Builder which will perform about as well as the Builder gem. Instead, use the lower-level Document and Node APIs.
Here's some code showing equivalent document construction for Builder, Nokogiri::XML::Builder, and Nokogiri::XML::Document ... and a benchmark if it's helpful (higher numbers are better, it's iterations per second):
```ruby #! /usr/bin/env ruby
require "bundler/inline"
gemfile do source "https://rubygems.org" gem "builder" gem "nokogiri" gem "benchmark-ips" end
require "builder" require "nokogiri" require "benchmark/ips"
N = 100
# using Builder::XmlMarkup, build an XML document representing attributes of a person def builder_person xml = Builder::XmlMarkup.new xml.people do |p| N.times do xml.person do |p| p.name "John Doe" p.age 42 p.address do |a| a.street "123 Main Street" a.city "Anytown" end end end end end
# using Nokogiri::XML::Builder, build an XML document representing attributes of a person def nokogiri_person Nokogiri::XML::Builder.new do |xml| xml.people do N.times do xml.person do |p| p.name "John Doe" p.age 42 p.address do |a| a.street "123 Main Street" a.city "Anytown" end end end end end.to_xml end
# use Nokogiri's bare API to build an XML document representing attributes of a person def nokogiri_raw_person xml = Nokogiri::XML::Document.new xml.root = xml.create_element("people") N.times do xml.root.add_child(xml.create_element("person")).tap do |p| p.add_child(xml.create_element("name", "John Doe")) p.add_child(xml.create_element("age", 42)) p.add_child(xml.create_element("address")).tap do |a| a.add_child(xml.create_element("street", "123 Main Street")) a.add_child(xml.create_element("city", "Anytown")) end end end xml.to_xml end
puts RUBY_DESCRIPTION # pp builder_person # pp nokogiri_person # pp nokogiri_raw_person
Benchmark.ips do |x| x.report("builder") { builder_person } x.report("nokogiri") { nokogiri_person } x.report("nokogiri raw") { nokogiri_raw_person } x.compare! end ```
``` ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux] Warming up -------------------------------------- builder 58.000 i/100ms nokogiri 49.000 i/100ms nokogiri raw 61.000 i/100ms Calculating ------------------------------------- builder 541.580 (± 6.5%) i/s - 2.726k in 5.054535s nokogiri 451.127 (± 7.5%) i/s - 2.254k in 5.027192s nokogiri raw 681.437 (± 9.1%) i/s - 3.416k in 5.061582s
Comparison: nokogiri raw: 681.4 i/s builder: 541.6 i/s - 1.26x slower nokogiri: 451.1 i/s - 1.51x slower ```
On Thu, May 18, 2023 at 8:02 AM Austin Ziegler via ruby-talk < ruby-talk@ml.ruby-lang.org> wrote:
I have never used the Rails builder interface for JSON or XML generation. It’s nice because it’s DSL-ish, but it needs to be executed.
If your data can be templated, using ERB will be better. If it is more complex with attributes, look into restructuring it so that you can use Nokogiri. I’m not sure if Ox generates as well as parses, but it may also be an option.
-a
On May 18, 2023, at 04:25, Peter Hickman via ruby-talk < ruby-talk@ml.ruby-lang.org> wrote:
I'm profiling some code to try and squeeze out performance. It takes a user query, reads data from the database and returned the result as XML
I have tuned the code and database (checking the indexes, caching repeated calls etc) and what I am left with is that 95% of the call's runtime is building the XML. We are using builder because it is the standard / default for Ruby
Using Ruby 3.0.2p107 as it is the default for Ubuntu 22.04
Is there another, faster library for XML generation or will I be building this by hand?
Any suggestions
______________________________________________ ruby-talk mailing list -- ruby-talk@ml.ruby-lang.org To unsubscribe send an email to ruby-talk-leave@ml.ruby-lang.org ruby-talk info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-talk.ml.ruby-lang.org...
ruby-talk mailing list -- ruby-talk@ml.ruby-lang.org To unsubscribe send an email to ruby-talk-leave@ml.ruby-lang.org ruby-talk info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-talk.ml.ruby-lang.org...
*Ruby users*
18. Mai 2023, 14:02 MESZ
I have never used the Rails builder interface for JSON or XML generation. It’s nice because it’s DSL-ish, but it needs to be executed.
If your data can be templated, using ERB will be better. If it is more complex with attributes, look into restructuring it so that you can use Nokogiri. I’m not sure if Ox generates as well as parses, but it may also be an option.
-a
On May 18, 2023, at 04:25, Peter Hickman via ruby-talk < ruby-talk@ml.ruby-lang.org> wrote:
I'm profiling some code to try and squeeze out performance. It takes a user query, reads data from the database and returned the result as XML
I have tuned the code and database (checking the indexes, caching repeated calls etc) and what I am left with is that 95% of the call's runtime is building the XML. We are using builder because it is the standard / default for Ruby
Using Ruby 3.0.2p107 as it is the default for Ubuntu 22.04
Is there another, faster library for XML generation or will I be building this by hand?
Any suggestions
______________________________________________ ruby-talk mailing list -- ruby-talk@ml.ruby-lang.org To unsubscribe send an email to ruby-talk-leave@ml.ruby-lang.org ruby-talk info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-talk.ml.ruby-lang.org...
ruby-talk mailing list -- ruby-talk@ml.ruby-lang.org To unsubscribe send an email to ruby-talk-leave@ml.ruby-lang.org ruby-talk info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-talk.ml.ruby-lang.org...
*Ruby users*
18. Mai 2023, 10:25 MESZ
I'm profiling some code to try and squeeze out performance. It takes a user query, reads data from the database and returned the result as XML
I have tuned the code and database (checking the indexes, caching repeated calls etc) and what I am left with is that 95% of the call's runtime is building the XML. We are using builder because it is the standard / default for Ruby
Using Ruby 3.0.2p107 as it is the default for Ubuntu 22.04
Is there another, faster library for XML generation or will I be building this by hand?
Any suggestions Avocado Store GmbH Cremon 32, 20457 Hamburg avocadostore.de <https://www.avocadostore.de> - Eco Fashion & Green Lifestyle [image: Instagram] <https://www.instagram.com/avocadostore.de> [image: Facebook] <https://de-de.facebook.com/avocadostore> [image: Pinterest] <https://pinterest.com/avocadostore> [image: TikTok] <https://www.tiktok.com/@avocadostore.de> [image: LinkedIn] <https://www.linkedin.com/company/avocadostore>
Eine E-Mail verursacht durchschnittlich 4 g CO2, mit Anhang bis zu 50 g. Jährlich bedeutet das 135 kg CO2-Emissionen pro Kopf. Verringere deine Emissionen, indem du nur notwendige E-Mails verschickst und regelmäßig E-Mails löschst. Registergericht: Amtsgericht Hamburg, HRB 113545 Umsatzsteuer-Identifikationsnummer: DE270706065 Geschäftsführung: Mimi Sewalski & Till Junkermann [QWV9ZW-K0GVP]
______________________________________________ ruby-talk mailing list -- ruby-talk@ml.ruby-lang.org To unsubscribe send an email to ruby-talk-leave@ml.ruby-lang.org ruby-talk info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-talk.ml.ruby-lang.org...