nyuu

Nyuu
====

> A powerful beast hidden by a deceptively cute exterior

Nyuu is a command-line binary usenet poster.

Nyuu is designed primarily to be fast/efficient, reliable and customizable,
exposing all the interesting bits and pieces. From what I’ve seen, Nyuu excels
more than any other usenet posting client in these aspects.

Nyuu runs on top of node.js (which everyone knows is bad ass rock star
tech) with minimal dependencies.
It does require one compiled module,
yencode, for ultra fast yEnc and
CRC32 calculation.

Features
========

All the standard stuff: i.e. Nyuu doesn’t miss out on what you need

- NNTP SSL support

- yEnc encoding

- NZB creation, with minification & compression

- Sub-directory handling

Fast & efficient: Nyuu is amongst the fastest (if not the
fastest) usenet posters, designed to run on low
power, overloaded servers with \>= 1Gbps connections

- Multiple uploading connections

- Post checking is asynchronous (minimal impact on speed)

- Fastest yEnc and CRC32 implementations

- Buffered async I/O (minimise the effects of slow disks)

- No temporary files, no disk writes (except for writing the NZB output)

- No unnecessary disk reads - only performs a single, sequential read pass
over files

- If a post needs to be resubmitted, due to a check failure, and it is not
cached, it will need to be re-read off disk

- PAR2 generation (when integrated) will require an extra initial pass of
just the first 16KB of each file, and may require multiple read passes
if all recovery blocks cannot fit in memory

Reliable: Nyuu was designed for automation and its reliability requirements

- Requests are all retryable

- Able to recover from connection failures/dropouts

- Timeouts and limits to deal with unexpected server hangs or faults

- Can selectively skip/ignore some errors

- Post checking (aka header checks), with multiple attempts and post retrying

- Unusual or unexpected events are logged

- Includes some optional workarounds for server bugs

Highly configurable: tuning knobs for everything

- Lots of connection and SSL options

- NNTP article posting: article/line sizes, header customisation

- Configure timeouts, delays and error handling parameters

- NZB meta tags

Unique features: the not so usual stuff

- Pipe input/output from/to processes instead of files, plus the ability to
pipe out a copy of read input to an external process without incurring
additional disk reads

- Extensive upload diagnostic details available (via optional TCP/HTTP status
server) to help tune settings or find problems

Installation & Requirements
===========================

Pre-Built Binaries
------------------

Static pre-built binaries with Node 4.x may be found on the releases
page if I can be bothered to
provide them. These can simply be extracted and run.

$3

Archlinux users can install the latest binary-release by installing nyuu-bin from the AUR.

Please note that this Package is maintained by a third party and not officially supported. If you have an issue please contact the maintainer.

Install Via NPM
---------------

If NPM is installed (usually comes bundled with
node.js), the following command can be used
to install Nyuu:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
npm install -g nyuu --production
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You’ll then be able to run Nyuu via the nyuu command.

If the npm command isn’t available, it can probably be installed via your
package manager (apt-get install npm for Debian), or see the following section
titled “Node.js” for more details.

If you get a gyp ERR! stack Error: EACCES: permission denied error when installing, try the following command instead:

``
npm install -g nyuu --production --unsafe-perm
`

Note: on some systems, you won’t get an error during installation, but when running Nyuu, it crashes with the error Error: Cannot find module './build/Release/yencode.node'. If that is the case, you may need to run the above command to get it to install properly.

You can then later uninstall Nyuu via:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
npm uninstall -g nyuu
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Install From Source
-------------------

Note that code from Git is considered to be unstable (or rather, less stable
than release versions). Stable packages can be found on the releases
page.

$3

Nyuu should run on node.js 0.10.x and later. Recent Linux distributions should
have nodejs in their repositories, otherwise, see installing via package
manager or
NodeSource. For Windows/OSX
builds, they can be found here.
Although node.js 0.10.x is supported, newer versions of Node (\>=4 recommended)
provide greatly improved SSL performance.

$3

If you have NPM installed (may come with your install of NodeJS, or you may need
to install it like you did with NodeJS if your package system doesn’t include
them together), the following command (executed inside the Nyuu directory) is
all you need to set up the dependencies:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
npm install --production
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you don’t want to use NPM, you can use your package manager instead if it has
all the necessary packages - on Debian 8 / Ubuntu 14.04:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
apt-get install node-async
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Alternatively, you can grab the dependencies manually: create a directory named
node_modules in the Nyuu directory. Inside of which, place
async.

For yencode, you’ll need to place it in the node_modules directory as well,
then follow the installation
instructions.

At the end, the folder structure should resemble something like (not all files
shown):

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Nyuu/
- bin/
- lib/
- node_modules/
- async/
- package.json
- yencode/
- index.js
package.json
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

$3

Due to security changes in OSX 10.15, libraries may require code signing to work. To deal with this, you’ll either need to disable this security option or codesign the built yencode module. Note that I do not have OSX and can’t provide much support for the platform.

$3

Once dependencies have been installed, Nyuu can be run via node bin/nyuu or
nodejs bin/nyuu. If you wish to just use nyuu instead, you need to link to
it from somewhere your PATH environment points to. For example, on Linux you
might do

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ln -s "pwd/bin/nyuu.js" /usr/bin/nyuu
chmod a+x bin/nyuu.js
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

(if Node is running via nodejs instead of node, you can edit the first line
in bin/nyuu.js to fix this)

For Windows, you can make a file named nyuu.cmd and place it in your system
directory, with the following contents:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@"C:\node\node.exe" "C:\path\to\nyuu\bin\nyuu.js" %*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

(obviously, fix the paths to what they actually are)

Development
===========

Running Tests
-------------

Tests are run via mocha, installable via npm install -g mocha, and can be
run simply by using the mocha command inside Nyuu’s root directory.
Note that some test cases test functionality of timeouts; to reduce time it
takes to run these tests, timeouts are set relatively small, which means that a
slow computer may not be able to service them as expected.

Building Binary
---------------

A basic script to compile the Nyuu binary is provided in the nexe folder. The script has been tested with NodeJS 12.20.0 and may work on other 12.x.x versions. For NodeJS 8.x.x and older, see section below.

1. If you haven’t done so already, do an npm install in Nyuu’s folder to ensure its dependencies are available
2. Enter the nexe folder and do an npm install to pull down required build packages (note, nexe requires NodeJS 10 or greater)
3. If desired, edit the variables at the top of nexe/build.js
4. Run node build. If everything worked, there’ll eventually be a nyuu or nyuu.exe binary built.
If it fails during compilation, enter the nexe/build/12.20.0 (or whatever version of NodeJS you’re using) and get more info by:
* Linux: build using the make command
* Windows: build using vcbuild.bat followed by build options, e.g. vcbuild nosign x86 noetw intl-none release static no-cctest without-intl ltcg
5. If building for Linux, optionally strip the executable to reduce size

$3

Compiling Nyuu into a single binary can be done via
nexe 1.x. There is a little complication with
bundling the yencode module, but a rather fragile script has been supplied in
nexe1/build.js to help with the process. The following general steps need to be
taken:

1. Ensure that nexe is installed (doesn’t need to be globally installed) and
its requirements met
2. Download a Node.js source package. The script has mostly been tested with
Node 4.x.x, it may work with other versions
3. The required Nyuu libraries need to be installed into the node_modules
folder
4. Inside the nexe1 folder (the one containing build.js), create the
following two folders: node and yencode-src
5. Inside the node folder, create a folder with the version number of the
package you downloaded in step 2, for example “4.9.1”. Inside this folder,
create one named “\_” and place the downloaded sources in this folder. After
doing this, the file nexe1/node/x.x.x/\_/node.gyp should exist, where
x.x.x is the node version number
6. Inside the yencode-src folder, copy the source code for the yencode (v1.0.x)
module
7. Edit nexe1/build.js; options that are likely to be edited are at the top of
the file. You’ll likely need to change nodeVer to be the version of node
you’re using
8. You may need to edit bin/nyuu.js to fix the location of nexe’s package.json path (search bin/nyuu.js for nexe/package.json and update if necessary)
9. In the nexe1 folder, run build.js. This script patches node to embed the
yencode module, and customises a few compiler options, then calls nexe to
build the final executable. If it worked, you should get a binary named
nyuu or nyuu.exe in the nexe1 folder

Note that this will be built with the -flto option on non-Windows platforms.
If this causes build failures, your system’s ar utility may not support LTO
objects, which can be worked around if you have gcc-ar installed by issuing a
command like export AR=gcc-ar

Usage
=====

For command line usage, see here, or here for a
summarized list of options.

Default Configuration
---------------------

As entering long command lines may be tiresome, you may use a config file with
Nyuu via the --config or -C option. You may use the config-sample.json
file as reference to what a config file should be like.

If a config file isn't specified, Nyuu will also search the NYUU_CONFIG`
environment variable for a default configuration (saving you from needing to
specify this file on every run, if you set the environment up in your shell).

Planned Features
================

- Integrate ParPar for streaming PAR2
creation

- Streaming 7-Zip creation

- Improve multi-server support for both posting and checking

- Repost missing articles from NZB, and/or some sort of resumption support

- SOCKS proxy support

- A web (HTTP) interface would be nice as an alternative to the command line
interface; not sure if it will be done however

Not planned
-----------

- RAR support, since 7-Zip should cover everything  

Alternatives
============

A list of Usenet posters I’ve come across can be found
here.

Here's a benchmark comparison between a few of the command line
posters.

License
=======

Nyuu is Public Domain or CC0 (or equivalent) if PD isn’t recognised. Use her as you will, at your own risk of course
(don’t come back crying if you lose a limb or two).