Populating the database - Revision history

Tros01: /* Discussion */

2025-01-26T22:53:18Z

‎Discussion

Tros01: /* Single run */

2025-01-26T22:47:39Z

‎Single run

Tros01: /* Single run */

2025-01-26T22:34:51Z

‎Single run

Tros01: /* Variables */

2025-01-26T22:16:20Z

‎Variables

Tros01: /* Discussion */

2025-01-20T00:00:37Z

‎Discussion

Tros01: /* Discussion */

2025-01-19T23:49:52Z

‎Discussion

Tros01: /* Single run */

2025-01-19T23:47:29Z

‎Single run

Tros01: /* Julia */

2025-01-19T23:40:09Z

‎Julia

Tros01: /* Model */

2025-01-19T23:38:12Z

‎Model

Tros01: /* Discussion */

2025-01-19T23:34:37Z

‎Discussion

@@ Line 208: / Line 208: @@
 *Enhance the feature of records quality so it could not be increased infinitely.
 *Add monitoring for values of records quality, so I can see better what is the state of the db like.
 =Code=

@@ Line 177: / Line 177: @@
 It seems that the first small batch upload has the largest influence on the resulting values and larger batches brings only relatively smaller improvements, that suggests to use the batch upload in real life, but its redundant to focus on optimizing the upload process for larger amounts, because they are not needed and only small amount of records (10) is enough to significantly boost the speed of populating.
-Also the amounts of records additionally created after 50 batch upload seems a bit unexpected, because after results seen in smaller batches one would expect a number around 15 or smaller. If it is not another bug in my code, I would guess that its bacause of small sample of results.
+Also the amounts of records additionally created after 50 batch upload seems a bit unexpected, because after results seen in smaller batches one would expect a number around 15 or smaller. If it is not another bug in my code, I would guess that its bacause of small sample of results. batch50[22, 27, 19, 22, 70], batch30[39, 25, 14, 24, 26] are the results obtained by additional runs, its still strange how much records is created in case of 50 batch, but also the cause of small sample is still viable explanation.
 ==Multi run==

← Older revision		Revision as of 22:34, 26 January 2025
Line 176:		Line 176:

	It seems that the first small batch upload has the largest influence on the resulting values and larger batches brings only relatively smaller improvements, that suggests to use the batch upload in real life, but its redundant to focus on optimizing the upload process for larger amounts, because they are not needed and only small amount of records (10) is enough to significantly boost the speed of populating.		It seems that the first small batch upload has the largest influence on the resulting values and larger batches brings only relatively smaller improvements, that suggests to use the batch upload in real life, but its redundant to focus on optimizing the upload process for larger amounts, because they are not needed and only small amount of records (10) is enough to significantly boost the speed of populating.
		+
		+	Also the amounts of records additionally created after 50 batch upload seems a bit unexpected, because after results seen in smaller batches one would expect a number around 15 or smaller. If it is not another bug in my code, I would guess that its bacause of small sample of results.

	==Multi run==		==Multi run==

@@ Line 68: / Line 68: @@
 ;NUM_AGENTS:
 : Integer representing the number of agents (users) participating in the simulation. Value is fixed at 3.
 ;BATCH_UPLOAD_SIZES:
-: Vector of integers representing different sizes of initial record batches uploaded to the database. The values used in simulation are [0, 10].
+: Vector of integers representing different sizes of initial record batches uploaded to the database. The values used in simulation are [0, 10, 30, 50].
 ;initialQualityDist:
 : A lognormal distribution (LogNormal(3, 2)) used to generate the initial quality of new records. The mean is set to 3, and the standard deviation is set to 2, meaning the typical quality of created record will be around 3, but there could be some with higher or lower.
 ;qualityImprovementProb
 : Float64 representing the probability that an agent will improve a record's quality after using it. Value is fixed at 0.7 for the sake of simplicity and simulation feasibility.
 ;recordCreationProb
 : Float64 representing the probability that an agent creates a new record when they are preparing an activity and decide not to search the database. Value is fixed at 0.3 for the sake of simplicity and simulation feasibility.
 ;categories
 : Vector of integer representing the category ids of the db records. The vector is generated as range from 1 to 20, [1:20;].
 ;targetSuccessfulness
 : Float64 value, which represents minimum success rate, that needs to be reached for the database to be considered "useful" in a long term. Value is 0.75.
@@ Line 84: / Line 98: @@
 ;episodeLength
 : Integer which represents length of a one "episode" within the simulation, which in this case is a sequence of weeks for which the success rate of the db will be tested if it met required conditions. Value is set to 6, this means 6 weeks.
 ==Flow==

@@ Line 186: / Line 186: @@
 Therefore the resuls appears to be approximately as they should, but they lack on the reliability if they should be used as a base for a decision.
 At most I can take a hint about the amount of records to batch upload. That relatively small amount makes a major difference.
 =Code=

← Older revision		Revision as of 23:49, 19 January 2025
Line 185:		Line 185:

	Therefore the resuls appears to be approximately as they should, but they lack on the reliability if they should be used as a base for a decision.		Therefore the resuls appears to be approximately as they should, but they lack on the reliability if they should be used as a base for a decision.
		+	At most I can take a hint about the amount of records to batch upload. That relatively small amount makes a major difference.

	=Code=		=Code=

@@ Line 159: / Line 159: @@
 It appears that the batch upload in the beginning realy speeds up the time required for the database to be considered usefull, but unfortunately the implementation can not ends earlier then after at least small fixed timeframe.
 Its obvious that the results are heavily influenced by the randomness in the model.
 ==Multi run==

@@ Line 39: / Line 39: @@
 Because the ABM approach was chosen the NetLogo was concidered as environment for implemantation of the simulation, but also was the language Julia as it is also often used for simulations. In the end the Julia was chosen because of its more general usability and therefore more usecases when learned to use.
-diskuse možností řešení, výběr metody a prostředku (metod a prostředků) řešení, zdůvodnění výběru (jinými slovy, proč to chcete řešit tak, jak to chcete řešit, jaké jsou jiné alternativy a proč je ta Vámi zvolená pro tuto úlohu nejvhodnější)
+The simulation is implemented in Julia using the Distributions, StatsBase, and Logging packages.
 =Model=

@@ Line 115: / Line 115: @@
 *The simulation continues until the database has reached "sufficient" usability.
 *A database is considered as "sufficiently useful" when success rates in a defined episode length (6 weeks) are over a defined threshold (75%) for a given minimum number of successful weeks (4 out of 6). In other words, when this condition is met within a 3 month timeframe.
 =Results=

@@ Line 170: / Line 170: @@
 =Discussion=
 In this simulation I managed to create heavily simplified model of database populating process in our boy scouts group.
 Therefore the resuls appears to be approximately as they should, but they lack on the reliability if they should be used as a base for a decision.