APS/NWNX released

Avlis Headlines - www.avlis.org

Moderator: Event DM

User avatar
Silk
Co-Founder
Posts: 6662
Joined: Fri Sep 14, 2001 6:47 pm
Contact:

Post by Silk » Mon Feb 03, 2003 12:12 am

We use it on the Mikona module which is over 50MB in size.
Are you talking about a larger module?
Silk

Member of the MadK@t lover's group.
Papillon
Team Member; Retired with Honors
Posts: 3155
Joined: Thu Jul 18, 2002 11:17 pm
Contact:

Post by Papillon » Mon Feb 03, 2003 12:27 am

Originally posted by Dregan
Hello there,

I have found a SERIOUS flaw with this system. Basically
the cpu usage is increased by a massive amount from the
standard server. This is related to the cpu usage of the
world to begin with. The system works quite excellent,
but is not usable on the very larger persistent world that
I work with. To verify this problem, a large MOD must be
used.
Well, this has been kind of answered already. I've your module is cranking out database queries like crazy, then you would definitely see increased CPU usage. If it just sits there idling, NWNX does the same and thus uses zero cpu cycles.
I have also located several smaller flaws in the APS system, and determined work-around. The first is a resolution to the problem mentioned earlier about finding 0 bytes for requests.
Yes, I coded a semaphore for that today. It'll be in the next release. I had hoped somebody would come up with it, but alas this was not the case. It's only 5 lines of code anyway...
Another problem is used the APS with the OnClientLeave function, where a player's object is no longer flagged as a PC and does not have a name or tag. A fix for this is to store a PC's name, tag, and PC status into a local variable, and only use local variables in the APS scripts.
I recommend using custom SQL statements in this case.

Hope I could help you.
Papillon
Team Member; Retired with Honors
Posts: 3155
Joined: Thu Jul 18, 2002 11:17 pm
Contact:

Post by Papillon » Mon Feb 03, 2003 12:30 am

Uhmm... I looked it up on nwvault. Are you talking about this really tiny mod with 7 MB ? That's not huge, that's a joke. I would give my left leg if Avlis was this small...*

* not that anybody would want it no the leg I mean
Dregan
Newbie
Posts: 5
Joined: Sun Feb 02, 2003 11:37 pm

CPU usage

Post by Dregan » Mon Feb 03, 2003 1:54 am

You are correct about the mod, as it is relatively small with about 30 areas. It doesn't even call the APS functions, or even initialize it. I simply ran it with the NWN Extender and compared it to the same module directly run from the nwserver process.

And in regards to the PW I work with, it has over 450 areas, and is a massive 77 MB mod, with a 600 MB Hak. With the NWNX stuff, it runs with twice the cpu usage.

Dave.
Dregan
Newbie
Posts: 5
Joined: Sun Feb 02, 2003 11:37 pm

CPU Usage

Post by Dregan » Mon Feb 03, 2003 3:01 am

Ok, it appears that the high cpu usage is due to the mod not actually having an SQL INIT function, which forces the NWNX program to continually search for the initializing sequence. So this is not a valid method of tracking CPU usage.

Regardless, the NWNX system has caused significant system cpu usage relative to not using it, with very light SQL calls on our PW. Something like 10 calls per minute.

Any advice here would be helpful.

Dave.
sowens
Newbie
Posts: 1
Joined: Mon Feb 03, 2003 4:00 am
Location: WPAFB, OHIO

So far so good

Post by sowens » Mon Feb 03, 2003 4:10 am

So far so good with converting the Module over to APS (with a lot of help). My one problem so far has been when my players exit the module. I seen that the question was somewhat answered above, but it is still unclear to me.

My SQL Statement is firing, but it is not pulling back any values. Can any suggest a solution and an example if possible. What do I do to get the location to save on exit for each character? It already saves at certain other times, but it fails on exit.....
slabar
Whiney Peasant
Posts: 14
Joined: Sun Feb 02, 2003 1:08 am
Contact:

Post by slabar » Mon Feb 03, 2003 4:24 am

Hello all. I don't mean to bother you but I haven't received any possible answers to my previous post. Question #1 I answered myself and have stored all old values and changed the scripts accordingly. However, I have no idea about questions 2 and 3 so I am reposting them. Thanks again for the help.

BTW Great job on the APS. I am running a 20M module with no lag and excellent access times.


2) I want to keep track of starting locations when the PC logs off or rests. How do I script that?

3) I would love to keep the world time from game to game. This is something I wish to do as we are only semi-persistent (playing once a week). How do I do this?

Thanks for both your help and your patience.
Papillon
Team Member; Retired with Honors
Posts: 3155
Joined: Thu Jul 18, 2002 11:17 pm
Contact:

Post by Papillon » Mon Feb 03, 2003 9:18 am

Originally posted by Dregan
Regardless, the NWNX system has caused significant system cpu usage relative to not using it, with very light SQL calls on our PW. Something like 10 calls per minute.

Any advice here would be helpful.

Dave.
I fear I can't help you here. I haven't received any reports on additional CPU usage by NWNX and I have no way to reproduce this problem. Let me reiterate: NWNX does absolutely nothing when it receives no requests, so I can't optimize anything here.
Papillon
Team Member; Retired with Honors
Posts: 3155
Joined: Thu Jul 18, 2002 11:17 pm
Contact:

Post by Papillon » Mon Feb 03, 2003 9:23 am

Guys, I'd love to help you all with your scripting problems, but unfortunately I don't have any time left (next to RL, NWNX, and Avlis) to answer scripting requests that are not APS/NWNX specific.

@sowens: Don't use the OnExit event. The player object isn't valid when this event is called.

@slabar: 2) try to get in contact with sowens and search together for a solution. Post it here once you found it.
3) I want to do the same, I'll let you know once I got around scripting this.
FastFrench
Whiney Peasant
Posts: 15
Joined: Wed Jan 29, 2003 10:42 am

Post by FastFrench » Mon Feb 03, 2003 9:56 am

As I got no answers yet, I try asking again. Maybe I'll be lucky this time...

1/ (29/1, on NWVault) do you think that your system may be even more efficient than NWN local variables and should also be used
to replace them (I think Bioware is really not good with
optimization matters... just have to consider how fast the toolset is ! So I imagine storing and retreiving datas is probably not well organized in NWN, and the fact NWServer allocates more and more memory from launch time 'til it crashes - In my 30Mbytes mod, NWServer starts with 35Mbytes at launch time, and has up to 400Mbyte at crash time, few hours later with 30-60 pple on)
Maybe with 'temporary' tables that would be cleaned at each restart (so to be sure database won't grow to much with junk)

2/ (29/1, on NWVault) what development tools did u use for NWN extender ?

3/ do you think you'll provide source code before or after Bioware's support for persistent data ?

Thanks for your answers :)
Papillon
Team Member; Retired with Honors
Posts: 3155
Joined: Thu Jul 18, 2002 11:17 pm
Contact:

Post by Papillon » Mon Feb 03, 2003 10:06 am

Originally posted by FastFrench

1/ (29/1, on NWVault) do you think that your system may be even more efficient than NWN local variables and should also be used
to replace them (I think Bioware is really not good with
optimization matters... just have to consider how fast the toolset is ! So I imagine storing and retreiving datas is probably not well organized in NWN, and the fact NWServer allocates more and more memory from launch time 'til it crashes - In my 30Mbytes mod, NWServer starts with 35Mbytes at launch time, and has up to 400Mbyte at crash time, few hours later with 30-60 pple on)
Maybe with 'temporary' tables that would be cleaned at each restart (so to be sure database won't grow to much with junk)
No.
2/ (29/1, on NWVault) what development tools did u use for NWN extender ?
(Visual) C++
3/ do you think you'll provide source code before or after Bioware's support for persistent data ?
This question has already been answered 25 times.

Sorry, maybe I've overlooked your questions.
Papillon
Team Member; Retired with Honors
Posts: 3155
Joined: Thu Jul 18, 2002 11:17 pm
Contact:

Post by Papillon » Tue Feb 04, 2003 9:56 am

I've uploaded an updated version of APS/NWNX to Neverwinter Vault (v.1.2, fixes potential startup problems).
FastFrench
Whiney Peasant
Posts: 15
Joined: Wed Jan 29, 2003 10:42 am

Post by FastFrench » Tue Feb 04, 2003 11:29 am

Well thanks for your answer Papillon.

Great choice for Q1, MSVC is my favourite tool ;)

For Q3, you clearly answered many time that you plan to release source code one day, but never seen any indication on when. So no idea if it has any chance to be before Bioware's PW support. Anyway I won't bug u anymore with this: you'll release when you'll feel ready to do it, I just hope it will still be of some use at that time.

Anyway keep doing such great tools. Even if I can't use it, you've clearly done a very good and useful piece of software.

FastFrench
[FR] !! FastFrench, Le bouchon Lyonnais ATS, PW Story
Papillon
Team Member; Retired with Honors
Posts: 3155
Joined: Thu Jul 18, 2002 11:17 pm
Contact:

Post by Papillon » Tue Feb 04, 2003 12:56 pm

Originally posted by FastFrench

For Q3, you clearly answered many time that you plan to release source code one day, but never seen any indication on when. So no idea if it has any chance to be before Bioware's PW support. Anyway I won't bug u anymore with this: you'll release when you'll feel ready to do it, I just hope it will still be of some use at that time.
That's because I don't know when it will be released. There will be a day when I feel it is stable enough to release the source.

Also, I don't see how anything that Bioware may do would affect APS/NWNX.
FastFrench
Whiney Peasant
Posts: 15
Joined: Wed Jan 29, 2003 10:42 am

Post by FastFrench » Tue Feb 04, 2003 6:36 pm

Originally posted by Papillon
Also, I don't see how anything that Bioware may do would affect APS/NWNX.
Oh, it probably won't probably affect APS/NWNX... will just affect me, as it may be a good enough solution that I could use to replace the @@@ tokens without giving up all monitoring tools I've alllready developped to keep NWServer running (I know APS/NWNX provides the most basic level of monitoring - restart when crash -, but that's clearly not enough).
User avatar
Silk
Co-Founder
Posts: 6662
Joined: Fri Sep 14, 2001 6:47 pm
Contact:

Post by Silk » Tue Feb 04, 2003 6:45 pm

@FastFrench: What other Monitoring Tool Functionality do you think would be useful or even necessary?
Silk

Member of the MadK@t lover's group.
Guest

Post by Guest » Wed Feb 05, 2003 8:30 am

@Silk: It's a bit complicate, but as it may be usefull for some others, here is what my monitoring tool do (Sometimes I'm up to 1 week away from the server, with no distant control on it. With usually 30-64 players online, such tool was clearly mandatory for me, even more important than good permanent datas. Also I'm the only DM on my server, and as I've my job + Toolset work, few time to come on the server...).
1/ Stop and restart NWServer.exe, when one of those happen:
- (1) when NWServer crash (of course)
- (2) when over 40 periods of 20 seconds with over 97% CPU usage
- (3) when has been FULL (>40 players) for at least 5 minutes, and is now empty, that is:
<5 players for at least 5 minutes,
or 0 players for at least 1 minute.
- (4) starting a counter at 30000, decrease with the number of active players every 20 seconds (x2 if over 87% CPU, x4 if over 95% CPU). When counter is down to 0 or down to 10000 if the server is empty (less than 5 players).
- (5) if allocated memory exceed 300 MB.
- (6) when asked to reload new update of the mod.

(note: NWServer.exe is stopped as cleanly as possible, to give him the best chance to save all chars, but still force kill if NWServer.exe can't close within 2 minutes, and retry several times with DrWatson checks).

2/ Detects and destroy any DrWatson thread, every 20 secondes.
(if this is not properly done, NWServer may still exist and be unkillable. I don't know why other don't do that... but other tools I've tried first clearly failed to do that, and so didn't work for me)

3/ Each time, before restarting NWServer, cleans the log file (remove most useless informations from it, keep like 10% of it), and add it to a big cumulative log file. This way I keep a complete archived history of all important events in my server (levels, PKs, steals, ATS technique usage infos, debug and stat infos...)

Note: can't understand how one can keep a NWN server running with just crash detection... must have a DM all the time to reload the mod when too much lag ?
Guest

Post by Guest » Wed Feb 05, 2003 8:32 am

(don't know why I didn't automatically registered... well, 'course was me, FastFrench.
Guest

Post by Guest » Wed Feb 05, 2003 8:46 am

Oh, I forgot another aspect:
4/ create stat binary files with: CPU, memory, Nb of users (min/max/mean for each period of 20 seconds). So I can graphically display it. That clearly show me that memory allocated grows all the time, speed of growing depends on number of users. This functionnality is not as usefull as it was at first, now that I have the conclusions I were looking for. Also it could be an independant tool, but in that case would require to check server infos twice, wich is not very well optimized.
5/ makes an history/log file with number of pple (add one line each time Nb of players changes, checking every 20 seconds), and each restart of the server with the reason of this restart (from all possible in 1/ and 4/ are the most common).

I probably still forgott some features, but that give you an idea...

FastFrench
perogi21
Newbie
Posts: 7
Joined: Mon Jan 20, 2003 4:44 pm

Dev from NWNOnline

Post by perogi21 » Wed Feb 05, 2003 8:00 pm

We are experiencing serious lag and jitter from using NWNX.
Couple of questions for the devs at Avlis:
1) Are you using MySQL for your database?
2) Whatever DB you are using, are you creating relational tables for data? IE. Are you using DB statements other than Set/GetPersistent*? Example: "Select * from tblPC WHERE PCID = 1" and requests like this?
3) Do you have the database on another server other than the ones running the mods?
4) Im sure that this is answered elsewhere but what hardware/OS etc are you using to run Avlis?

Thanks for any insight!
Pele.

p.s. Papillon, any ideas??
*edit*
(Our mod is 65MB)
*end edit*
slabar
Whiney Peasant
Posts: 14
Joined: Sun Feb 02, 2003 1:08 am
Contact:

Post by slabar » Wed Feb 05, 2003 10:37 pm

perogi21,

I know this probrably won't help much be here's what I'm doing. I have a 22 Mb Module running semi-persistent. I am running NWN on a Windows 2000 Server. I am using a MySQL for my database and have that on my web server (cobalt raq4r) located out of state. We have played (only one day mind you) for over 7 hours with no lag.

I hope that helps a little bit.
Pelemele
Whiney Peasant
Posts: 15
Joined: Mon Jan 20, 2003 3:54 pm

Post by Pelemele » Wed Feb 05, 2003 11:28 pm

slabar,

How are you calling the Set/Get persistent functions? Just what they have in aps_include? Or did you make your own functions?

Thanks,
Pele.

aka perogi21
slabar
Whiney Peasant
Posts: 14
Joined: Sun Feb 02, 2003 1:08 am
Contact:

Post by slabar » Thu Feb 06, 2003 6:47 am

Pelemele,

As of right now I am using the included (aps_include) functions in my scripts. Two reasons for this one I am only an average scripter, and two I am currently trying to figure out how to get the Players location on both resting and when they log off.

Just a simple script I'm using to update Journals on entry.

if (GetPersistentInt(oPC, "questbalgar") == 2)
{
AddJournalQuestEntry("Murichamson", 20, oPC, bAllPlayers);
}

Hope that helps you a little bit. Holler if I can be more helpful.
perogi21
Newbie
Posts: 7
Joined: Mon Jan 20, 2003 4:44 pm

Post by perogi21 » Thu Feb 06, 2003 3:57 pm

Getting information from a PC that is logging off is spotty at best but I can probably help you with the get/set location on rest.

<the following code is straight from my head. No guarantees that it will compile>


object oPC;
oPC = GetLastPCRested();
location lLoc;

lLoc = GetLocation(oPC);

SetPersistentString(oPC, "Loc"+GetName(oPC)+GetKey(oPC), LocationToString(lLoc));

(Use those special wrapper methods around the GetName and GetKey to prevent corrupting the data with a " ' ". Above and below.)

To get the location:
location lLoc;
string sLocation;
sLocation = GetPersistentString(oPC, "Loc"+GetName(oPC)+GetKey(oPC));
lLoc = StringToLocation(sLocation);

Simple eh?
Pele.

p.s. We are not using these methods to load/extract data. I am hoping that the devs will give me a heads up with what they do in this world.
slabar
Whiney Peasant
Posts: 14
Joined: Sun Feb 02, 2003 1:08 am
Contact:

Post by slabar » Fri Feb 07, 2003 6:43 am

Papillon wrote:Guys, I'd love to help you all with your scripting problems, but unfortunately I don't have any time left (next to RL, NWNX, and Avlis) to answer scripting requests that are not APS/NWNX specific.

@sowens: Don't use the OnExit event. The player object isn't valid when this event is called.

@slabar: 2) try to get in contact with sowens and search together for a solution. Post it here once you found it.
3) I want to do the same, I'll let you know once I got around scripting this.
Papillon,

I have a script for the keeping of time from game to game. You will probrably find it rough (not a great scripter) and will undoubtably tweak it. If the tweaks make it better please post them so I may look at it.

1) I had to create a custom object (named mine time_sundial) - you can probrably use any object for this

2) placed this script in the Module ClientOnExit

[nwscript]

#include "aps_include"

void main()
{
object oSundial = GetObjectByTag ("time_sundial");

int iYear = GetCalendarYear();
int iMonth = GetCalendarMonth();
int iDay = GetCalendarDay();
int iHour = GetTimeHour();

SetPersistentInt(oSundial, "Month", iMonth);
SetPersistentInt(oSundial, "Day", iDay);
SetPersistentInt(oSundial, "Hour", iHour);
SetPersistentInt(oSundial, "Year", iYear);

}

[/nwscript]

3) Placed this script in the Module ClientOnEnter

[nwscript]
#include "aps_include"

void main ()
{

object oPC = GetFirstPC();

object oSundial = GetObjectByTag ("time_sundial");

int nYear = GetPersistentInt(oSundial, "Year");
int nMonth = GetPersistentInt(oSundial, "Month");
int nDay = GetPersistentInt(oSundial, "Day");

int nHour = GetPersistentInt(oSundial, "Hour");
int nMinute = 0;
int nSecond = 0;
int nMillisecond = 0;


location lLoc;
string sLocation;
sLocation = GetPersistentString( oPC, "Location" + GetName(oPC));
lLoc = StringToLocation(sLocation);

ActionJumpToLocation (lLoc);

SendMessageToPC(oPC, "This is an entering object test.");

SetCalendar (nYear, nMonth, nDay );
SetTime (nHour, nMinute, nSecond, nMillisecond);

}

[/nwscript]

Hope that's what you were looking for and it helps you.

Now for my question (had to know I'ld have one :lol: ).

You will note in the above OnEnter script I am trying to jump my PC's to their last resting place but it's not working. I have tried various oPC calls to no avail. Can someone please tell me what I'm doing wrong.

Thanks for the help.

[edit]
Of course I did forget to mention that I am going to add a RUNONCE argument to the OnEnter script. Sorry about that.
[/edit]
Post Reply